GavinW_29074
Dec 23, 2014Nimbostratus
HTTP Payload logging breaks HTTP Keep-Alive
Afternoon all,
I've written an iRule to record the request/response payload on a REST HTTP API.
The rule looks like: when CLIENT_ACCEPTED {
if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "Processing CLIENT_ACCEPTED." }
Set the payload logging flag
set log_payload 1
}
when HTTP_REQUEST priority 800 {
if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Processing HTTP_REQUEST at priority 800..." }
Skip logging if no members available
if {$splunk_bypass}{
if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Splunk HSL pool is down. Bypassing logging..." }
return
}
Don't allow data to be chunked
if { [HTTP::version] eq "1.1" } {
if { [HTTP::header is_keepalive] } {
HTTP::header replace "Connection" "Keep-Alive"
}
if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Switching to HTTP Version 1.0." }
HTTP::version "1.0"
}
Split out request headers and munge into string
set headers [HTTP::header names]
if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Request HTTP Headers = $headers" }
set request_headers "'"
foreach header $headers {
set value [HTTP::header value $header]
if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Request HTTP Header $header value = $value" }
append request_headers "$header=$value "
}
set request_headers [string trimright $request_headers " "]
append request_headers "'"
}
when HTTP_REQUEST_DATA {
if { $static::PayloadLoggerDebug or $f5_connection_debug }{ log local0.debug "$log_prefix: Collected [HTTP::payload length] bytes."}
set request_payload [HTTP::payload]
}
when HTTP_RESPONSE priority 50 {
if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Processing HTTP_RESPONSE at priority 50..." }
Skip logging if no members available
if {$splunk_bypass}{
if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Splunk HSL pool is down. Bypassing logging..." }
return
}
Split out the response headers and munge into string
set headers [HTTP::header names]
if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Response HTTP Headers = $headers" }
set response_headers "'"
foreach header $headers {
set value [HTTP::header value $header]
if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Response HTTP Header $header value = $value" }
append response_headers "$header=$value "
}
set response_headers [string trimright $response_headers " "]
append response_headers "'"
Collect the response
if { $response_length > 0 } {
if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Collecting $response_length bytes from response." }
HTTP::collect $response_length
}
Calculate actual content-length
set response_length_real [HTTP::payload length]
if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Real response content-length = $response_length_real." }
Correct the response_length to correct value if required.
if { $response_length != $response_length_real } {
if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Updated \$response_length value." }
set response_length $response_length_real
if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: New \$response_length value is $response_length." }
}
}
when HTTP_RESPONSE_DATA {
if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Processing HTTP_RESPONSE_DATA." }
Gather response data
set response_payload "[HTTP::payload]"
}
A lot of the above feeds into a larger iRule framework. E.g. we use a SplunkHTTPS iRule to actually do the HSL logging out to a syslog server. The 'HTTP::collect' is also called from that iRule aswell.
However when testing this iRule, I've identified an issue whereby it appears to be breaking HTTP Keep-alive connections to some of our GF3 application servers.
Removing this rule restores the keep-alive functionality.
Any pointers on how I can maintain the keep-alive functionality and also be able to log the request/response data?
Cheers Gavin