Log file

Two most common logging daemons: rsyslogd, journald.

Configure: /etc/rsyslog.conf, /etc/logrotate.conf.

To configure log files override the common configuration in logrotate.conf file use the fomate:

/path/filename {
conditions
}

To verify the configuration file:

sudo rsyslogd -N1

path and filename specify the log file you want to configure, conditions can be follows:

Use man logrotate (http://linux.die.net/man/8/logrotate) for more information:

• Rotate count

The “rotate” command determines how many archived logs will be kept around before logrotate starts deleting the older ones. For example:

rotate 4

That command tells logrotate to keep 4 archived logs at a time. If there are already four archived logs when the log is rotated again, the oldest one (the one with “.4” at the end, usually) will be deleted to make room for the new archive.

• Rotation interval

You can specify a command that will tell logrotate how often to rotate a particular log. The possible commands include:

daily
weekly
monthly
yearly

If a rotation interval is not specified the log will be rotated whenever logrotate runs (unless another condition like “size” has been set).

If you want to use a time interval other than the keywords listed here you’ll have to get clever with cron and a separate config file. For example, if you wanted to rotate a particular log file hourly, you could create a file in “/etc/cron.hourly” (you may need to create that directory too) that would contain a line like:

/usr/sbin/logrotate /etc/logrotate.hourly.conf

Then put the configuration for that hourly run of logrotate (the log file location, whether or not to compress old files, and so on) into “/etc/logrotate.hourly.conf”.

• Size

You can specify a file size that logrotate will check when determining whether or not to perform a rotation by using the “size” command. The format of the command tells logrotate what units you’re using to specify the size:

size 100k
size 100M
size 100G

The first example would rotate the log if it gets larger than 100 kilobytes, the second if it’s larger than 100 megabytes, and the third if it’s over 100 gigabytes. I don’t recommend using a limit of 100G, mind you, the example just got a little out of hand there.

The size command takes priority over and replaces a rotation interval if both are set.

minsize sizeLog files are rotated when they grow bigger than size bytes, but not before the additionally specified time interval (daily, weekly, monthly, or yearly). The related size option is similar except that it is mutually exclusive with the time interval options, and it causes log files to be rotated without regard for the last rotation time. When minsize is used, both the size and timestamp of a log file are considered.

• Compression

If you want archived logfiles to be compressed (in gzip format) you can include the following command, usually in /etc/logrotate.conf:

compress

This is normally a good idea, since log files are usually all text, and text compresses very well. You might, however, have some archived logs you don’t want compressed, but still want compression to be on by default. In those cases you can include the following command in an application-specific config:

nocompress

One more command of note in regard to compression is:

delaycompress

This command can be useful if you want the archived logs to be compressed, but not right away. With “delaycompress” active an archived log won’t be compressed until the next time the log is rotated. This can be important when you have a program that might still write to its old logfile for a time after a fresh one is rotated in. Note that “delaycompress” only works if you also have “compress” in your config.

• missingok : If the log file is missing, go on to the next one without issuing an error message.
• mail address : When a log is rotated out-of-existence, it is mailed to address. If no mail should be generated by a particular log, the nomail directive may be used.

 

Give each host / Group of host a dedicated log file

The /var/log/message will grow fast if we have a bunch of network devices sending syslog to the linux server. Then it’s better to create a dedicated log file for each host or a similar group of hosts.

All we need to do is craft a ” if then ” sentence in the /etc/rsyslog.conf file:

 # do this in FRONT of the local/regular rules
 if $fromhost-ip startswith '192.0.1.' then /var/log/network1.log
 & ~
 if $fromhost-ip startswith '192.0.2.' then /var/log/network2.log
 & ~
 # local/regular rules, like
 *.* /var/log/syslog.log

How it works:

The if’s above check if a message originates on the network in question and, if so, writes them to the appropriate log. The next line (“& ~”) is important: it tells rsyslog to stop processing the message after it was written to the log. As such, these messages will not reach the local part. Without that “& ~”, messages would also be written to the local files. So we add the rule in front of the regular rules.

Also note that in the filter there is a dot after the last number in the IP address. This is important to get reliable filters. For example, both of the addresses “192.0.1.1″ and “192.0.10.1″ start with “192.0.1″ but only one actually starts with “192.0.1.”!

Forward cisco router syslog to fedora server:

Device: R1 which model is Cisco Router C3745(192.168.0.2) or others, Fedora 20 64bit(192.168.0.66) or other linux server.

Step 1, Issue ifconfig on the to check the ip address of the Fedora, in this example is 192.168.0.66. Log on R1, ping 192.168.0.66, make sure you can ping the fedora server.

Step 2, Let the router forward the syslog to fedora server.

R1(config)#logging trap debugging

R1(config)# logging 192.168.0.66     // or use your own Linux server’s IP

Step 3, Configure the Linux syslog server.

The configuration file is /etc/rsyslog.conf, Use VI or other text editor to open it.

For the rsyslog service to listen to UDP port 514, you have to uncomment the following lines:

$ModLoad imudp
$UDPServerRun 514

For saving all incoming local7 logging data in a separate file, you should add the following line at the end of the /etc/rsyslog.conf file:

local7.* /var/log/cisco.log

On CentOS 7 the default location is

/var/log/messages

Create the cisco.log file and give it write permission:

# touch /var/log/cisco.log

# chmod 666 /var/log/cisco.log

You should now restart the rsyslog service for changes to take effect. You can restart the service as follows:

# service rsyslog restart

Step4. enable firewall to accept the UDP port 514

Click activity, search Firewall, add new port 514, protocol type UDP.

Step5. make sure the Port 514 is open by using NMAP.

On fedora 20, NMAP is installed, if not,use  sudo yum install nmap.x86_64

Use command sudo nmap -sU localhost to check the ports open on Linux server, make sure the line 514/udp open|filtered syslog

is there.

Step 6. Verify

Log on cisco router, use command router#send log 4 warning!!warning!! to create a syslog entry at level 4.

Switch to the Fedora server, then issue command tail -5 /var/log/cisco.log

Syslog-ng and Splunk

 

syslog-ng.org

Setup splunk:

default location: /opt/splunk/

To start : /opt/splunk/bin/splunk start

 

Splunk 是一个运行于 Unix 环境下的日志分析软件.与 Google Analytics 这一类的 Web 日志分析软件的不同之处在于，Splunk 可以支持任何服务器产生的日志，其对日志进行处理的方式是进行高效索引之后让管理员可以对日志中出现的各种情况进行搜索，并且通过非常好的图形化的方式展现出来。

每天由各种服务器所产生的日志的数量是非常惊人的，而遇到突发情况时，却往往能够从这些海量日志中找到最多的有用消息。通常在 Unix 下对日志进行查找使用的是 grep 之类的低效率的方式，而 Splunk 使用了现代搜索引擎技术对日志进行搜索，同时提供了一个非常强大的 AJAX 式的界面展现日志(文字来自互联网)。

需关闭selinux

vi /etc/sysconfig/selinux 
SELINUX=disabled 
setenforce 0

开始安装

wget http://download.splunk.com/releases/3.4.9/linux/splunk-3.4.9-57762-Linux-i686.tgz 
tar -zxvf splunk-3.4.9-57762-Linux-i686.tgz 
cd .. 
mv splunk /usr/local 
cd /usr/local/splunk/bin 
./splunk enable boot-start 
/etc/init.d/splunk start

 

netstat -an 
0.0.0.0:8000 
0.0.0.0:8089

监听以上两个端口则完成安装。

界面截图

整合syslog

添加syslog-ng步骤：

在Admin->Data Inputs->Network Ports上点选New Input.选择TCP 9998 port, Set Source Type选Form List,Source Type选Syslog,这样的设定就可以给Syslog-ng传log了。

添加syslog步骤：

再重复一次添加syslog-ng的步骤，但是端口要改成UDP 514就可以了。