How does ELK works

ELK stack provides a powerful platform to index, search and analyze your data. It uses  Logstash for log aggregation, Elasticsearch for searching, and Kibana for visualizing and analyzing data. In short, ELK stack:

  • Collect logs and events data (Logstash)
  • Make it searchable in fast and meaningful ways (Elasticsearch)
  • Use powerful analytics to summarize data across many dimensions (Kibana)

elk1

Install ELK

Note that you have to install the consistent version of all the stack software, eg. 5.02 elasticsearch will not work with 5.22 Kibana.

Here I will take the latest version 5.2.2 (as of 2017/03/27) as example.

Install the Elasticsearch
Prerequisite:

Elasticsearch requires Java 8 or later. Use the official Oracle distribution or an open-source distribution such as OpenJDK.

Import the Elasticsearch PGP Key

We sign all of our packages with the Elasticsearch Signing Key (PGP key D88E42B4, available from https://pgp.mit.edu) with fingerprint:

4609 5ACC 8548 582C 1A26 99A9 D27D 666C D88E 42B4

Download and install the public signing key:

rpm –import https://artifacts.elastic.co/GPG-KEY-elasticsearch

Installing from the RPM repository

Create a file called elasticsearch.repo in the /etc/yum.repos.d/ directory for RedHat based distributions, or in the /etc/zypp/repos.d/ directory for OpenSuSE based distributions, containing:

[elasticsearch-5.x]
name=Elasticsearch repository for 5.x packages
baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

And your repository is ready for use. You can now install Elasticsearch :

sudo yum install elasticsearch
Manual install
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.2.2.rpm
sha1sum elasticsearch-5.2.2.rpm 
sudo rpm -iv elasticsearch-5.2.2.rpm

Preparing packages...
Creating elasticsearch group... OK
Creating elasticsearch user... OK
elasticsearch-0:5.2.2-1.noarch
### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd
 sudo systemctl daemon-reload
 sudo systemctl enable elasticsearch.service
### You can start elasticsearch service by executing
 sudo systemctl start elasticsearch.service

Then we enable the service start on system startup and start the service:

[root@localhost elasticsearch]# sudo systemctl daemon-reload
[root@localhost elasticsearch]# sudo systemctl enable elasticsearch.service
Created symlink from /etc/systemd/system/multi-user.target.wants/elasticsearch.service to /usr/lib/systemd/system/elasticsearch.service.
[root@localhost elasticsearch]# sudo systemctl start elasticsearch.service

To check the logging info of systemd:

To tail the journal:

sudo journalctl -f

To list journal entries for the elasticsearch service:

sudo journalctl --unit elasticsearch

To list journal entries for the elasticsearch service starting from a given time:

sudo journalctl --unit elasticsearch --since  "2016-10-30 18:17:16"

If you are not sure the location of the configuration file, check it in the process:

[root@localhost elasticsearch]# ps -ef | grep elasticsearch
elastic+ 101064      1 38 10:53 ?        00:00:12 /bin/java -Xms2g -Xmx2g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly 
-XX:+DisableExplicitGC -XX:+AlwaysPreTouch -server -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -Djdk.io.permissionsUseCanonicalPath=true 
-Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true 
-Dlog4j.skipJansi=true -XX:+HeapDumpOnOutOfMemoryError -Des.path.home=/usr/share/elasticsearch -cp /usr/share/elasticsearch/lib/elasticsearch-5.2.2.jar:/usr/share/elasticsearch/lib/* 
org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/elasticsearch.pid --quiet -Edefault.path.logs=/var/log/elasticsearch -Edefault.path.data=/var/lib/elasticsearch 
-Edefault.path.conf=/etc/elasticsearch
root     101154 100478  0 10:54 pts/3    00:00:00 grep --color=auto elasticsearch

The important info here is:

  • default.path.conf=/etc/elasticsearch
  • default.path.logs=/var/log/elasticsearch
  • Des.path.home=/usr/share/elasticsearch

 

Pages: 1 2 3 4