Syslog Monitoring with Opsview

In this guide, I will show you a quick and easy way to get open source syslog monitoring using Opsview. Migrating from Nagios to Opsview is easy due to Opsview being fully compatible with Nagios plugins.
This guide will use the built-in Linux syslog server ‘rsyslog’ and a great plugin called ‘check_logfiles’.

Step 1: The basics

What you need to do is quite simple – you send your syslogs to the syslog server (Opsview, in this example) – and then you run a plugin / script against the central log file and check for just ‘HostX’ syslogs, and just for certain patterns against HostX. I.e. you send your router's syslogs to Opsview. You also want to check for the word ‘CRITICAL’ in that routers log files – as below:

Step 2: Setting up the syslog server

Firstly, you need to configure rsyslog on the Opsview server. Edit the file rsyslog.conf and uncomment the section at the top:

# Provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514

As above, we are going to be using UDP in this example (I’ve found that my Draytek doesn’t like playing with TCP syslog servers, hmm!). Next, give it a restart – “service rsyslog restart” and that’s that configured.

Finally, open up a port on the firewall using a rule such as:

iptables -I INPUT 3 -p udp --dport 514 -j ACCEPT

You can add a “-s” rule if your only allowing a certain subnet, also.

Step 3: Configure the syslog client

On the network device(s), you want to configure it to send the syslogs to the Opsview server. In this example we are using a Draytek router and as such configure it as below:

As above, we are pointing the router to send syslogs to the Opsview server (192.168.0.15) at port 514 (the one that was allowed through the firewall earlier).

That’s all that is required – you can tweak the different messages to your preference as above, but in this example we’ll leave everything ticked bar VPN and Call Log.

Step 4: Check logs are received

This is simple, run a command such as:

tail –f /var/log/messages | grep hostname

Where ‘hostname’ is your devices name, i.e. smnet_router as above:

[root@localhost sam]# tail -f /var/log/messages | grep smnet_router
Jul 29 11:17:39 192.168.0.254 smnet_router: [FW_QoS][Class1][@S:R=13:1, 192.168.0.12:5356->192.168.0.25:8080]
Jul 29 11:17:39 192.168.0.254 smnet_router: [FW_QoS][Class1][@S:R=13:1, 192.168.0.12:5356->192.168.0.25:8080]
Jul 29 11:17:39 192.168.0.254 smnet_router: [FW_QoS][Class1][@S:R=13:1, 192.168.0.12:5356->192.168.0.25:8080]
Jul 29 11:17:39 192.168.0.254 smnet_router: [FW_QoS][Class1][@S:R=13:1, 192.168.0.12:5356->192.168.0.25:8080]
Jul 29 11:17:39 192.168.0.254 smnet_router: [FW_QoS][Class1][@S:R=13:1, 192.168.0.12:5356->192.168.0.25:8080]
Jul 29 11:17:39 192.168.0.254 smnet_router: [FW_QoS][Class1][@S:R=13:1, 192.168.0.12:5356->192.168.0.25:8080]

As you can see above, the router is now logging to the Opsview syslog server – terrific!

Step 5: Begin analysis

Next, you can begin to carve up your log files and analyse them using a multitude of methods such as “check_logs” or “check_logfiles” (one of the better open source ones around).
I won’t bore you with the specifics, however you can download the package from the website here: http://labs.consol.de/lang/en/nagios/check_logfiles/
You then “configure, make and install” as you would normally, and then run “check_logfiles” against your newly gathered log files (read the documentation provided in the link for more information).
Essentially, once you’ve built the plugin – copy it and the .pl version to /usr/local/nagios/libexec – and then chmod +x it and also chown nagios:nagios it:

chmod +x check_logfiles*
chown nagios:nagios check_logfiles*

This ensures it is now available via the Opsview GUI to be used. You can then configure it however you want. The official blurb says:
“For the simplest applications it is sufficient to call check_logfile with command line parameters. More complex scan jobs can be described with a config file.”
So basically, run something similar to:

check_logfiles –tag smnet_router –logfile=/var/log/messages –criticalpattern=”CRITICAL”

Where –tag hostname is your device, i.e. smnet_router; and “CRITICAL” is the string you want to alert on. In the example below, we have chosen a fairly common string just to demonstrate the plugin is working:Where –tag hostname is your device, i.e. smnet_router; and “CRITICAL” is the string you want to alert on. In the example below, we have chosen a fairly common string just to demonstrate the plugin is working:

[nagios@localhost libexec]$ ./check_logfiles --tag=smnet_router --logfile=/var/log/messages --criticalpattern="FW_Session"
CRITICAL - (494 errors in check_logfiles.protocol-2013-07-29-11-23-11) - Jul 29 11:23:10 192.168.0.254 smnet_router: [FW_Session][Pass][320/60000][@S:R=13:1, 192.168.0.12:5409->192.168.0.16:9890] ...|smnet_router_lines=3382 smnet_router_warnings=0 smnet_router_criticals=494 smnet_router_unknowns=0
[nagios@localhost libexec]$

Step 6: Bring it all together

Finally, you can take the plugin and tests from the CLI and add them into your Opsview GUI as below:

You can even potentially replace “–tag=smnet_router” with “–tag=$HOSTADDRESS$”, so that you can apply the Opsview methodology of ‘create once, apply many times’ – this saves you having to create 1000 service checks for 1000 devices just to check for the word ‘CRITICAL’ for example.