Syslog.

The most common method of accessing system messages that networking devices provide is to use a protocol called syslog.

Syslog is a term used to describe a standard. It is also used to describe the protocol developed for that standard. The syslog protocol was developed for UNIX systems in the 1980s, but was first documented as RFC 3164 by IETF in 2001.

Syslog uses UDP port 514 to send event notification messages across IP networks to event message collectors.

Many networking devices support syslog, including: routers, switches, application servers, firewalls, and other network appliances. The syslog protocol allows networking devices to send their system messages across the network to syslog servers.

There are several different syslog server software packages for Windows and UNIX. Many of them are freeware.

The syslog logging service provides three primary functions:

  • The ability to gather logging information for monitoring and troubleshooting
  • The ability to select the type of logging information that is captured
  • The ability to specify the destinations of captured syslog messages

On Cisco network devices, the syslog protocol starts by sending system messages and debug output to a local logging process internal to the device.

These messages can be retrieved without the need of accessing the actual device. Log messages and outputs stored on the external server can be pulled into various reports for easier reading.

Alternatively, syslog messages may be sent to an internal buffer. Messages sent to the internal buffer are only viewable through the CLI of the device.

Finally, the network administrator may specify that only certain types of system messages are sent to various destinations. For example, the device may be configured to forward all system messages to an external syslog server. However, debug-level messages are forwarded to the internal buffer and are only accessible by the administrator from the CLI.

  • Logging buffer (RAM inside a router or switch)
  • Console line
  • Terminal line
  • Syslog server
Syslog message format

The smaller numerical levels are the more critical syslog alarms. The severity level of the messages can be set to control where each type of message is displayed (i.e. on the console or the other destinations). The complete list of syslog levels:

Severity Name Severity Level Explanation
Emergency Level 0 System Unusable
Alert Level 1 Immediate Action Needed
Critical 2 Critical condition
Error 3 Error Condition
Warning 4 Warning Condition
Notification 5 Normal, but significant Condition
Informational 6 Information Message
Debugging 7 debugging message

Note: You can remember the order above with the sentence: “Eventually All Critical Errors Will Not Involve Damage”.

Each syslog level has its own meaning:

  • Warning Level – Emergency Level – These messages are error messages about software or hardware malfunctions; these types of messages mean that the functionality of the device is affected. The severity of the issue determines the actual syslog level applied.
  • Debugging Level – This level indicates that the messages are output generated from issuing various debug commands.
  • Notification Level – The notifications level is only for information, device functionality is not affected. Interface up or down transitions, and system restart messages are displayed at the notifications level.

In addition to specifying the severity, syslog messages also contain information on the facility. Syslog facilities are service identifiers that identify and categorize system state data for error and event message reporting. The logging facility options that are available are specific to the networking device. For example, Cisco 2960 Series switches running Cisco IOS Release 15.0(2) and Cisco 1941 routers running Cisco IOS Release 15.2(4) support 24 facility options that are categorized into 12 facility types.

Some common syslog message facilities reported on Cisco IOS routers include:

  • IP
  • OSPF protocol
  • SYS operating system
  • IP security (IPsec)
  • Interface IP (IF)

By default, the format of syslog messages on the Cisco IOS Software is as follows:

seq no: timestamp: %facility-severity-MNEMONIC: description

Field Explanation
seq no Stamps log messages with a sequence number only if the service sequence-number global configuration command is configured
 timestamp Date and time of the message or event, which appears only if the service timestamps global configuration command is configured
 facillity the facillity to which the message refers
 severity Single-digit code from 0 to 7 that is the severity of the message
 MNEMONIC text string that uniquely describes the message
description Text string containing detailed information about the event being reported
Timestamp

Log messages can be time-stamped and the source address of syslog messages can be set. This enhances real-time debugging and management.

When the service timestamps log uptime global configuration mode command is entered, the amount of time since the switch last booted is displayed on logged events.

A more useful version of this command applies the datetime keyword in place of the uptime keyword;service timestamps log datetime , this forces each logged event to display the date and time associated with the event.

When using the datetime keyword, the clock on the networking device must be set. This can be accomplished in one of two ways:

  • Manually set, using the clock set command
  • Automatically set, using the Network Time Protocol (NTP)

Enable millisecond (msec) timestamps using the service timestamps command:

 

 router(config)#service timestamps debug datetime msec 
      router(config)#service timestamps log datetime msec

These commands add time stamps to debugs in the format MMM DD HH:MM:SS, indicating the date and time according to the system clock. If the system clock has not been set, the date and time are preceded by an asterisk (*) to indicate that the date and time are probably not correct.

It is generally advisable to configure millisecond timestamps as this provides high level of clarity when looking at debug outputs. Millisecond timestamps provide a better indication of the timing of the various debugs events relative to each other. However, note that, when the console port outputs a lot of messages, they might not correlate with the actual timing of the event. For example, if you enable debug x25 all on a box that has 200 VCs, and the output is logged to the buffer (using the no logging console and logging buffered commands), the timestamp displayed in the debug output (within the buffer) might not be the exact time when the packet passes through the interface. Therefore, do not use msec timestamps to prove performance issues, but to obtain relative information on when events occur.

Configure NTP

Recall that NTP is a protocol that is used to allow network devices to synchronize their time settings with an NTP server.

To allow the software clock to be synchronized by an NTP time server, use the ntp server ip-address  command in global configuration mode.

Use ntp master number to set a router as  NTP server.

Configuring Syslog

 

To view syslog messages, a syslog server must be installed on a workstation in the network. Eg. kiwi syslog server.

The syslog server provides a relatively user-friendly interface for viewing syslog output. The server parses the output and places the messages into pre-defined columns for easy interpretation. If timestamps are configured on the networking device sourcing the syslog messages, then the date and time of each message displays in the syslog server output.

By default, Cisco routers and switches send log messages for all severity levels to the console.

On some IOS versions, the device also buffers log messages by default. To enable these two settings, use the logging console and logging buffered global configuration commands, respectively.

The show logging command displays the default logging service settings on a Cisco router.

On most Cisco IOS routers, the default severity level is 7, debugging.

Show_logging

The output also notes that 32 such messages have been logged.

The second highlighted line states that this router logs to an internal buffer. Because this router has enabled logging to an internal buffer, the show logging command also lists the messages in that buffer.

configure syslog client

There are three steps to configuring the router to send system messages to a syslog server where they can be stored, filtered, and analyzed:

Step 1. Configure the destination hostname or IP address of the syslog server in global configuration mode:

R1(config)# logging 192.168.1.3

Step 2. Control the messages that will be sent to the syslog server with the logging trap level global configuration mode command. For example, to limit the messages to levels 4 and lower (0 to 4), use one of the two equivalent commands:

R1(config)#   logging trap 4
R1(config)#   logging trap warning

Step 3. Optionally, configure the source interface with the logging source-interface interface-type interface number global configuration mode command. This specifies that syslog packets contain the IPv4 or IPv6 address of a specific interface, regardless of which interface the packet uses to exit the router. For example, to set the source interface to g0/0, use the following command:

R1(config)# logging source-interface g0/0

Verify syslog

use show logging command, some pipe options:

show logging | include changed state
show logging | begin June 12

if you want to create a syslog message entry without shutting down a interface or other action, simply use the syntax:

send log [severity] [text to send]

eg: send log 4 Warning!!Warning!!!

Issue

timestamp showing the UTC time instead of local time:

service timestamps log datetime localtime show-timezone

Security

Cisco routers can log information regarding configuration changes, ACL violations, interface status, CPU utilization, and many other types of events.

For example, use the memory free low-watermark threshold io and memory free low-watermark processor commands to set memory thresholds. The router will send notifications, specified in kilobytes to the syslog server when available free memory falls below the threshold. The router will send notifications again when the available free memory rises to five percent above the threshold.