Set Up Filebeat (Add Client Servers)
Do these steps for each CentOS or RHEL 7 server that you want to send logs to your ELK Server. For instructions on installing Filebeat on Debian-based Linux distributions (e.g. Ubuntu, Debian, etc.), refer to the Set Up Filebeat (Add Client Servers) section of the Ubuntu variation of this tutorial.
Copy SSL Certificate(optional-only do this step if )
On your ELK Server, copy the SSL certificate—created in the prerequisite tutorial—to your Client Server (substitute the client server’s address, and your own login):
- scp /etc/pki/tls/certs/logstash-forwarder.crt user@client_server_private_address:/tmp
After providing your login’s credentials, ensure that the certificate copy was successful. It is required for communication between the client servers and the ELK Server.
Now, on your Client Server, copy the ELK Server’s SSL certificate into the appropriate location (/etc/pki/tls/certs):
- sudo mkdir -p /etc/pki/tls/certs
- sudo cp /tmp/logstash-forwarder.crt /etc/pki/tls/certs/
Now we will install the Topbeat package.
Install Filebeat Package
On Client Server, create run the following command to import the Elasticsearch public GPG key into rpm:
- sudo rpm –import http://packages.elastic.co/GPG-KEY-elasticsearch
Create and edit a new yum repository file for Filebeat:
- sudo vi /etc/yum.repos.d/elastic-beats.repo
Add the following repository configuration:
- [beats]
- name=Elastic Beats Repository
- baseurl=https://packages.elastic.co/beats/yum/el/$basearch
- enabled=1
- gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
- gpgcheck=1
Save and exit.
Install Filebeat with this command:
- sudo yum -y install filebeat
Filebeat is installed but it is not configured yet.
Configure Filebeat
Now we will configure Filebeat to connect to Logstash on our ELK Server. This section will step you through modifying the example configuration file that comes with Filebeat. When you complete the steps, you should have a file that looks something like this.
On Client Server, create and edit Filebeat configuration file:
- sudo vi /etc/filebeat/filebeat.yml
Note: Filebeat’s configuration file is in YAML format, which means that indentation is very important! Be sure to use the same number of spaces that are indicated in these instructions.
Near the top of the file, you will see the prospectors section, which is where you can define prospectors that specify which log files should be shipped and how they should be handled. Each prospector is indicated by the - character.
We’ll modify the existing prospector to send secure and messages logs to Logstash. Under paths, comment out the - /var/log/*.log file. This will prevent Filebeat from sending every .log in that directory to Logstash. Then add new entries for syslog and auth.log. It should look something like this when you’re done:
old version
...
paths:
- /var/log/secure
- /var/log/messages
# - /var/log/*.log
...
(Optional, If there is no document_type keyword in your configuration file, skip this)Then find the line that specifies document_type:, uncomment it and change its value to “syslog”. It should look like this after the modification:
...
document_type: syslog
...
This specifies that the logs in this prospector are of type syslog (which is the type that our Logstash filter is looking for).
If you want to send other files to your ELK server, or make any changes to how Filebeat handles your logs, feel free to modify or add prospector entries.
Next, under the output section, find the line that says elasticsearch:, which indicates the Elasticsearch output section (which we are not going to use). Delete or comment out the entire Elasticsearch output section (up to the line that says logstash:).
Find the commented out Logstash output section, indicated by the line that says #logstash:, and uncomment it by deleting the preceding #. In this section, uncomment the hosts: ["localhost:5044"] line. Change localhost to the private IP address (or hostname, if you went with that option) of your ELK server:
### Logstash as output
logstash:
# The Logstash hosts
hosts: ["ELK_server_private_IP:5044"]
This configures Filebeat to connect to Logstash on your ELK Server at port 5044 (the port that we specified an input for earlier).
(Optional)Directly under the hosts entry, and with the same indentation, add this line:
bulk_max_size: 1024
(Optional) Next, find the tls section, and uncomment it. Then uncomment the line that specifies certificate_authorities, and change its value to ["/etc/pki/tls/certs/logstash-forwarder.crt"]. It should look something like this:
...
tls:
# List of root certificates for HTTPS server verifications
certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]
This configures Filebeat to use the SSL certificate that we created on the ELK Server.
New version in 5.22:
filebeat.prospectors: # Each - is a prospector. Most options can be set at the prospector level, so # you can use different prospectors for various configurations. # Below are the prospector specific configurations. - input_type: log # Paths that should be crawled and fetched. Glob based paths. paths: - /var/log/secure - /var/log/messages # - /var/log/*.log
Outputs section:
#================================ Outputs ===================================== # Configure what outputs to use when sending the data collected by the beat. # Multiple outputs may be used. #-------------------------- Elasticsearch output ------------------------------ #output.elasticsearch: # Array of hosts to connect to. # hosts: ["localhost:9200"] # Optional protocol and basic auth credentials. #protocol: "https" #username: "elastic" #password: "changeme" #----------------------------- Logstash output -------------------------------- output.logstash: #The Logstash hosts hosts: ["localhost:5044"] # Optional SSL. By default is off. # List of root certificates for HTTPS server verifications #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"] # Certificate for SSL client authentication #ssl.certificate: "/etc/pki/client/cert.pem" # Client Certificate Key #ssl.key: "/etc/pki/client/cert.key"
Save and quit.
Now start and enable Filebeat to put our changes into place:
- sudo systemctl start filebeat
- sudo systemctl enable filebeat
Again, if you’re not sure if your Filebeat configuration is correct, compare it against this example Filebeat configuration.
Now Filebeat is sending your syslog messages and secure files to your ELK Server! Repeat this section for all of the other servers that you wish to gather logs for.
Test Filebeat Installation
If your ELK stack is setup properly, Filebeat (on your client server) should be shipping your logs to Logstash on your ELK server. Logstash should be loading the Filebeat data into Elasticsearch in an date-stamped index, filebeat-YYYY.MM.DD.
On your ELK Server, verify that Elasticsearch is indeed receiving the data by querying for the Filebeat index with this command:
- curl -XGET ‘http://localhost:9200/filebeat-*/_search?pretty’
You should see a bunch of output that looks like this:
...
{
"_index" : "filebeat-2016.01.29",
"_type" : "log",
"_id" : "AVKO98yuaHvsHQLa53HE",
"_score" : 1.0,
"_source":{"message":"Feb 3 14:34:00 rails sshd[963]: Server listening on :: port 22.","@version":"1","@timestamp":"2016-01-29T19:59:09.145Z","beat":{"hostname":"topbeat-u-03","name":"topbeat-u-03"},"count":1,"fields":null,"input_type":"log","offset":70,"source":"/var/log/auth.log","type":"log","host":"topbeat-u-03"}
}
...
If your output shows 0 total hits, Elasticsearch is not loading any logs under the index you searched for, and you should review your setup for errors. If you received the expected output, continue to the next step.
Connect to Kibana
When you are finished setting up Filebeat on all of the servers that you want to gather logs for, let’s look at Kibana, the web interface that we installed earlier.
If you want to connect to Kibana from another client, make sure the TCP port 80, 5044 are open:
firewall-cmd --add-port=80/tcp --permanent firewall-cmd --add-port=5044/tcp --permanent
In a web browser, go to the FQDN or public IP address of your ELK Server.
After entering the “kibanaadmin” credentials, you should see a page prompting you to configure a default index pattern:
Reference
Reference_ Directory structure:
https://www.elastic.co/guide/en/logstash/current/dir-layout.html
https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elk-stack-on-centos-7
Install Logstash: https://www.elastic.co/guide/en/logstash/current/installing-logstash.html
For a more sophisticated system with Redis:
http://www.everybodyhertz.co.uk/setting-up-a-relk-stack-a-how-to/