How to Configure SNMPv3 Traps

We have various articles already in our documentation for setting up SNMPv2 trap handling in Opsview, but SNMPv3 traps are a whole new ballgame. They can be quite confusing and complicated to set up the first time you go through the process, but when you understand what is going on, everything should make more sense.

About SNMP

SNMP has gone through several revisions to improve performance and security (version 1, 2c and 3). By default, it is a UDP based protocol where communication is based on a 'fire and forget' methodology in which network packets are sent to another device, but there is no check for receipt of that packet (versus TCP when a network packet must be acknowledged by the other end of the communication link). 

There are two modes of operation with SNMP - get requests (or polling) where one device requests information from an SNMP enabled device on a regular basis (normally using UDP port 161), and traps where the SNMP enabled device sends a message to another device when an event occurs (normally using UDP port 162). The latter includes instances such as someone logging on, the device powering up or down, or a wide variety of other problems that would need this type of investigation.

This blog covers SNMPv3 traps, as polling and version 2c traps are covered elsewhere in our documentation.

SNMP traps

Since SNMP is primarily a UDP based system, traps may be 'lost' when sending between devices; the sending device does not wait to see if the receiver got the trap. This means if the configuration on the sending device is wrong (using the wrong receiver IP address or port) or the receiver isn't listening for traps or rejecting them out of hand due to misconfiguration, the sender will never know.

The SNMP v2c specification introduced the idea of splitting traps into two types; the original 'hope it gets there' trap and the newer 'INFORM' traps. Upon receipt of an INFORM, the receiver must send an acknowledgement back. If the sender doesn't get the acknowledgement back, then it knows there is an existing problem and can log it for sysadmins to find when they interrogate the device.

SNMPv3 

Older versions of SNMP relied on a single unencrypted "community string" for both get requests and traps, which made it very insecure on the network (anyone with network experience could 'snoop' on the network and detect the unencrypted community strings). The only security options with SNMP v1 and v2c are to either disable it altogether (not very helpful for monitoring purposes) or make sure SNMP enabled devices are 'read only' (SNMP can allow you to update configuration on devices) so that if the connection details were obtained by a 3rd party, they could only look at what was happening rather than change device configuration. However, even that approach can provide too much information to the wrong type of party.

Version 3 uses the same base protocol as earlier versions, but introduces encryption and much improved authentication mechanisms. Depending on how you authorize with the SNMP agent on a device, you may be granted different levels of access.

The security level you use depends on what credentials you must provide to authenticate successfully:

Security Level	Description	secName	authProtocol	authPassword	privProtocol	privPassword
noAuthNoPriv	No authentication and no encryption	Required	Not Required	Not Required	Not Required	Not Required
authNoPriv	Messages are authenticated but not encrypted	Required	Required	Required	Not Required	Not Required
authPriv	Messages are authenticated and encrypted	Required	Required	Required	Required	Required

 

As can be seen above, the best security scheme is 'authPriv', but it takes more to configure a working system.

For systems to communicate, both sides must use the same authProtocol (MD5 or SHA) and privProtocol (AES or DES). Some devices do not support all of these combinations, so you must check what can be used to ensure the trap receiver is configured in the same way.

If you have multiple devices that need to use multiple schemes, you should create multiple users, one for each scheme type. Do not try to shoe-horn all devices into using one type as this makes managing the systems harder than necessary. But the fewer schemes you need to set up on the SNMP devices, the less configuration and password management you need on the receiver.

Version 3 also introduces the concept of an 'SNMP Engine ID', which is a unique identifier for each SNMP device usually expressed as a hexadecimal string such as '0x8000123acd1ab43abbfff000fa'. If your SNMP sender has an engine ID, this must be taken into account when creating the authentication schemes on the receiver (note: engine ID's are not used for INFORMs).

Setting up SNMPv3 traps on an Opsview server

Start by creating a SNMPv3 user in the trap originator host. To do so you will need to configure the:

• the username
• the authprotocol (MD5 / SHA) and authpassphrase
• the privprotocol (DES / AES) and privpassphrase

Next you need to add the IP address of the Opsview collector that will monitor the trap originator host, this can either be the master collector or a collector belonging to a cluster.

On the receiver side (i.e. the Opsview collector) repeat the configuration, this time adding the SNMPv3 user to the snmptrapd.conf file (generally located on /etc/snmp/snmptrapd.conf). In addition to the username, authprotocol / authpassphrase and privprotocol / privpassphrase it is necessary to define the engine ID of the snmptrap sender - unless the trap is being sent as an INFORM. To identify the engine ID run the following command on the trap originator host:

snmpget -v1 -Cf -c public localhost 1.3.6.1.6.3.10.2.1.1.0

If the MIB specified in the command above is not available you can download it from this link.

With the previous information we can now add the user to the snmptrapd.conf file:

createUser -e <engine1d> <username> <authprotocol> <authpassphrase> <privprotocol> <privpassphrase>

By default, snmptrapd must be told what to do with traps received from authenticated users, therefore we need to add the following line to the snmptrad.conf file:

authUser log,execute <username> 

Without this line, the trap is received but nothing else will be done with it, not even logged.

Finally we need to have the incoming traps be processed by Opsview, to achieve this add the following line:

traphandle default /opt/opsview/snmptraps/bin/stdin2sock

Restart snmptrapd on the Opsview collector to enable the configurations. 

sudo service snmptrapd restart

Example:

If the following SNMPv3 user exists on the trap originator host:

createUser opsviewv3 SHA mySecureAuthPassword AES mySecurePrivPassword

Then on the Opsview collector that is monitoring that host we should have the following configuration on the snmptrapd.conf file:

createUser -e 0x8000123acd1ab43abbfff000fa opsviewv3 SHA mySecureAuthPassword AES mySecurePrivPassword
authUser log,execute opsviewv3

disableAuthorization yes
traphandle default /opt/opsview/snmptraps/bin/stdin2sock

As noted above, if the trap is being sent as an INFORM the engine -e 0x8000123acd1ab43abbfff000fa can be omitted.

Note: On some systems when the snmptrap daemon is started, it may remove the createUser lines from the snmptrapd.conf file and convert them to usmUser lines, so don't worry if what you have configured seems to disappear.

Troubleshooting SNMPv3

There are many ways to troubleshoot SNMPv3 issues, but these are the main methods we recommend: 

First, confirm the receiver is getting the traps sent through. You can do this using a network snooper command as root:

tcpdump -vv -A -T snmp -s 0 "(dst port 161) or (dst port 162) and (host <sender IP address>)"

This will look for all network packets sent to either port 161 or 162 on the local machine from the senders IP address. You may have an option on the sender to generate a test trap, in which case you should see some activity from the tcpdump command (use <CTRL>-C to stop the trace). If you see traffic here, then you can move onto the next step. Otherwise, check firewalls between the sender and receiver to make sure traffic can be passed through.

The second stage would be to stop snmptapd from running as a daemon and restart it in the foreground with debug logging enabled. Again, run the daemon, send the test trap, wait a few seconds, then stop the daemon using <CTRL>-C and go through the log file to see if your trap has been received and if so, what was done with it:

snmptrapd -t -m ALL -Oa -M /usr/share/snmp/mibs:/opt/opsview/var/snmp -p /var/run/snmptrapd.pid -f -Lo -DALL -Lf /tmp/snmptrap.out

You can also send a test trap from the receiver into itself pretending to be the sending device, which may help in your debugging process to make sure traps can be received and processed:

snmptrap -e <engineID> -v3 -u <secName> -a <authProtocl> -A <authPassword> -x <pricProtocol> -X <privPassword> -l <scheme> localhost 1 0

snmptrap -e 0x8000123acd1ab43abbfff000fa -v 3 -u opsviewv3 -a SHA -A mySecureAuthPassword -x AES -XmySecurePrivPassword -l authPriv localhost 1 0

If you need to test an INFORM instead of a standard trap, use the following: 

snmpinform -v 3 -u opsviewv3 -a SHA -A mySecureAuthPassword -x AES -XmySecurePrivPassword -l authPriv localhost 1 0

Summary

SNMP is by no means simple. However, it can be very useful when it is set up properly and to do that you need some understanding of how it all fits together. Hopefully the above will provide you with what you need and feel free to read more information about SNMP traps on our Knowledge Center or via a Net-SNMP tutorial.

Also, be sure to learn more about how Opsview can help you with your SNMP monitoring needs.