Forum Discussion

Jeff_Kerzner_16's avatar
Jeff_Kerzner_16
Icon for Nimbostratus rankNimbostratus
Dec 24, 2014

Splunk Log Publisher with iRule HSL

Hi,

I'm having some difficulty understanding the relationship between the HSL iRule commands and formatted log publishers (Splunk, in this case) in 11.5.1. Sorry if the formatting isn't what the community expects - I'm fairly new to this ecosystem.

Setup
  • One node called "splunk-universal-forwarder" running a Splunk universal forwarder listening on 9996/tcp.

  • A pool called "remote-logging-pool" with the splunk-universal-forwarder node listening on 9996/tcp.

  • Log Destination "splunk-rhsl" of type Remote HSL pointed at the remote-logging-pool over TCP.

  • Log Destination "splunk-formatted" of type "Splunk" forwarding to "splunk-rhsl"

  • Log Publisher "splunk-publisher" pointing at the splunk-formatted destination.

  • Log Publisher "rhsl-publisher" pointing at the splunk-rhsl destination.

  • A virtual server backed by a pool with a single node serving up content http://10.1.1.2 (the IP address is irrelevant for this question).

Given the iRule:

when CLIENT_ACCEPTED {
    set hsl [HSL::open -publisher /tst/splunk-publisher]
   set hsl [HSL::open -proto TCP -pool remote-logging-pool]
   set hsl [HSL::open -publisher /tst/Rhsl-publisher]
}
when HTTP_REQUEST {
  HSL::send $hsl "<190>|[IP::local_addr]|[HTTP::uri]\n"
}

The bottom two

set
statements in the
when CLIENT_ACCEPTED
command result in a messaging going successfully to the Splunk forwarder.

The first

set
statement results in no messages going to the Splunk forwarder.

Questions
  1. Are HSL commands in iRules unable to use formatted publishers?
  2. What exactly does a formatted publisher do? It seems like it would write data to the remote receiver in a format that particular receiver expects, but I haven't found a lot of documentation on this yet.
  3. Is there an interactive way to generate dummy log events to send to different destinations and publishers to see what the effect is on remote receiver?

Thanks for your help!

  • I don't think using a formatted publisher for an iRule will work. I say that because HSL::send is sending in whatever format you put in between the quotes for the part of the command. Formatted publishers are intended for system logs, HTTP request logging profiles, AVR, DNS logging profiles, etc... While I haven't seen that written, that is how I understand the functionality from my experience.

     

  • https://clouddocs.f5.com/api/irules/HSL__open.html

    HSL::open -publisher <publisher>

    • Opens and returns a handle for High Speed Logging communication for a log publisher configured in System->Logs->Configuration->Log Publishers. The handle should be used with the HSL::send command to send data to the publisher. introduced in v11.3

     

    When deciding on a publisher for your log messages, bear in mind that HSL::send will not work if the publisher is configured with some formatted destinations like arcsight or splunk. If the log server expects CEF or Splunk formatted messages, the iRule should craft the data the way the server expects it to be formatted and send to a publisher configured with an unformatted destination, such as remote-high-speed-log.