I’ve decided to switch to Splunk for my syslog parser. I was using Syslog Watcher, however I realized that I need something that I can customize to correctly parse the data coming from pfSense. The reason being is because of the way pfSense generates the firewall events. The output is split into two lines instead of one; this format causes a problem with many popular syslog & SIEM applications. Combining the lines into one will be required if you want to do proper analytics & reporting. With just a few tweaks, Splunk handles the parsing the way I want it to.
I run Splunk 6.1 Enterprise on a CentOS server. The RPMs can be found on splunk.com.
Here is a link to a quick youtube video that gives a quick run through of the installation.
Installing Splunk 6 on Linux (CentOS, Red Hat).
A few commands and things to note:
$SPLUNK_HOME/bin/splunk start <stop, restart>
Configure Splunk to start at boot time with a startup (init) script
$SPLUNK_HOME/bin/splunk enable boot-start
Splunk runs on default port 8000
How to configure Splunk to handle pfSense data
This is the really cool thing about Splunk. It is the ultimate SIEM application in terms of customization. There are two config files that give you the ability to parse the data and output it the way you want:
props.conf – allows Splunk to recognize the multi-line pfSense events as one.
transforms.conf – the parsing of the data received into the fields that you want to see.
Full credit goes to this blog for the awesome regex tailor-made to parse pfSense.
- Check that pfSense is configured to send log messages to remote syslog server.
- From the Splunk Web GUI go to Settings – Data inputs – UDP.
This is where Splunk is configured to listen on UDP 514 (syslog). Here are my custom settings:
Now you can go to the App: Search & Reporting and you will see your indexed data. Click the Data Summary button and it will launch a window where you can view the various sources that Splunk is listening to.
4. You can use the search field to customize your search. The results can be saved as a report, dashboard or alert. My query displays in table format showing the fields: _time, src_ip, src_port, dest_ip, dest_port, protocol, action.
The result is pretty neat compared to reading the raw data format.
Here are some helpful Splunk links for Search. I still have some playing around to do to create some nice visually appealing charts and reports. I plan on making some custom search queries to cover various time periods such as: 24hr, week, month and year, to make it easy to pull statistics and perform analytics.
Happy Splunking 🙂