4x Affordable, 99.95% SLA, 24x& Video Support, 100+ Countires

Install Graylog2 And Centralize Logs On Ubuntu 14 04

Introduction

In this tutorial, we will cover the installation of Graylog2 (v0.20.2), and configure it to collect the syslogs of our systems in a centralized area. Graylog2 is a mighty log management and analysis equipment that has many use cases, from observing SSH logins and different activity to debugging applications. It is based on Elasticsearch, Java, MongoDB, and Scala.

Note: This tutorial is for an outdated version of Graylog2. A new version is available here: How To Install Graylog 1.x on Ubuntu 14.04.

It is feasible to use Graylog2 to accumulate and observe a gigantic show of logs, but we will maximum the extent of this tutorial to syslog collecting . Also, because we are showing the fact Synonyms/Hypernyms of Graylog2, we will be installing all of the elements on an individual server.

About Graylog2 Components

Graylog2 has four important elements:

  • Graylog2 Server nodes: Serves as a worker that receives and processes messages, and communicates with all other non-server elements. Its performance is CPU babelike
  • Elasticsearch nodes: accumulations all of the logs/messages. Its performance is thrust and disk I/O babelike
  • MongoDB: accumulations metadata and does not experience much load
  • Web interface: The user interface

Here is a drawing of the Graylog2 elements (note that the messages are sent from your other servers):

Basic Graylog2 Setup

For a very basic setup, all of the elements can be installed on the same server. For a large, production setup, it would be wise to set up some high-availability features because if the server, Elasticsearch, or MongoDB elements experiences an outage, Graylog2 will not collect the messages generated during the outage.

Prerequisites

The setup described in this tutorial requires an ubuntu 14.04 VPS with at least 2GB of thrust. You also need set accesses (stages 1-4 of first Server Setup with Ubuntu 14.04).

If you use a vps with less than 2GB of thrust you will not be able to commence all of the Graylog2 elements.

Let's commence installing program!

Install MongoDB

The MongoDB installation is uncomplicated and fast. Run the following command to import the MongoDB public GPG important into inclined:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10

Create the MongoDB source database:

echo 'deb http://downloads-distro.mongodb.org/repo/debian-sysvinit dist 10gen' | sudo tee /etc/apt/sources.list.d/mongodb.list

modify your inclined package database:

sudo apt-get update

Install the current stable model of MongoDB with this command:

sudo apt-get install mongodb-org

MongoDB should be up and running now. Let's move on to installing Java 7.

Install Java 7

Elasticsearch requires Java 7, so we will install that now. We will install Oracle Java 7 because that is what is recommended on elasticsearch.org. It should, however, work satisfactory with OpenJDK, if you decide to go that path.

increase the Oracle Java PPA to inclined:

sudo add-apt-repository ppa:webupd8team/java

modify your inclined package database:

sudo apt-get update

Install the current stable model of Oracle Java 7 with this command (and accept the license agreement that pops up):

sudo apt-get install oracle-java7-installer

Now that Java 7 is installed, let's install Elasticsearch.

Install Elasticsearch

Graylog2 v0.20.2 requires Elasticsearch v.0.90.10. Download and install it with these regulates:

cd ~; wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.90.10.deb
sudo dpkg -i elasticsearch-0.90.10.deb

We need to action the Elasticsearch agglomeration.name setting. ajar the Elasticsearch configuration register:

sudo vi /etc/elasticsearch/elasticsearch.yml

Find the part that specifies agglomeration.name. Uncomment it, and replace the failure ideal with "graylog2", so it looks like the following:

agglomeration.name: graylog2

You will also want to restrict outside accesses to your Elasticsearch example (port 9200), so visitors can't read your data or shutdown your Elasticseach agglomeration through the HTTP API. Find the line that specifies network.bind_host and uncomment it so it looks like this:

network.bind_host: localhost

Then increase the following line somewhere in the register, to disable non-stative scripts:

script.disable_dynamic: true

Save and quit. Next, restart Elasticsearch to put our actions into effect:

sudo service elasticsearch restart

After a few seconds, run the following to experiment that Elasticsearch is running properly:

curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'

Now that Elasticsearch is up and running, let's install the Graylog2 server.

Install Graylog2 server

Now that we have installed the other demanded app, let's install the Graylog2 server. We will install Graylog2 Server v0.20.2 in /opt. First, download the Graylog2 archive to /opt with this command:

cd /opt; sudo wget https://github.com/Graylog2/graylog2-server/releases/download/0.20.2/graylog2-server-0.20.2.tgz

Then extract the archive:

sudo tar xvf graylog2-server-0.20.2.tgz

Let's create a symbolic link to the newly created directory, to simplify the directory name:

sudo ln -s graylog2-server-0.20.2 graylog2-server

Copy the instance configuration register to the proper area, in /etc:

sudo cp /opt/graylog2-server/graylog2.conf.example /etc/graylog2.conf

Install pwgen, which we will use to generate password confidential keys:

sudo apt-get install pwgen

Now we must configure the admin password and confidential important. The password confidential important is configured in graylog2.conf, by the password_secret parameter. We can generate an ergodic important and attach it into the Graylog2 configuration with the following two regulates:

SECRET=$(pwgen -s 96 1)
sudo -E sed -i -e 's/password_secret =.*/password_secret = '$SECRET'/' /etc/graylog2.conf

The admin password is assigned by creating an shasum of the desired password, and assigning it to the set_password_sha2 parameter in the Graylog2 configuration register. Create shasum of your desired password with the following command, substituting the highlighted "password" with your own. The sed command inserts it into the Graylog2 configuration for you:

PASSWORD=$(echo -n password | shasum -a 256 | awk '{print $1}')
sudo -E sed -i -e 's/set_password_sha2 =.*/set_password_sha2 = '$PASSWORD'/' /etc/graylog2.conf

Now that the admin password is setup, let's ajar the Graylog2 configuration to make a few actions:

sudo vi /etc/graylog2.conf

You should see that password_secret and set_password_sha2 have ergodic necklaces to them, because of the regulates that you ran in the levels above. Now we will configure the rest_transport_uri, which is how the Graylog2 web interface will communicate with the server. Because we are installing all of the elements on a solo server, let's set the ideal to 127.0.0.1, or localhost. Find and uncomment rest_transport_uri, and action it's ideal so it looks like the following:

rest_transport_uri = http://127.0.0.1:12900/

Next, because we only have one Elasticsearch piece (which is running on this server), we will action the ideal of elasticsearch_shards to 1:

elasticsearch_shards = 1

Save and quit. Now our Graylog2 server is configured and prepared to be began .

elective: If you want to experiment it out, run the following command:

sudo java -jar /opt/graylog2-server/graylog2-server.jar --debug

You should see a lot of production. Once you see production akin to the following lines, you will know that your Graylog2 server was configured correctly:

2014-06-06 14:16:13,420 INFO : org.graylog2.Core - Started REST API at <http://127.0.0.1:12900/>
2014-06-06 14:16:13,421 INFO : org.graylog2.Main - Graylog2 up and running.

Press CTRL-C to kill the experiment and return to the shell.

Now let's install the Graylog2 init script. Copy graylog2ctl to /etc/init.d:

sudo cp /opt/graylog2-server/bin/graylog2ctl /etc/init.d/graylog2

modify the startup script to put the Graylog2 logs in /var/log and to look for the Graylog2 server JAR register in /opt/graylog2-server by running the two following sed regulates:

sudo sed -i -e 's/GRAYLOG2_SERVER_JAR=\${GRAYLOG2_SERVER_JAR:=graylog2-server.jar}/GRAYLOG2_SERVER_JAR=\${GRAYLOG2_SERVER_JAR:=\/opt\/graylog2-server\/graylog2-server.jar}/' /etc/init.d/graylog2
sudo sed -i -e 's/LOG_FILE=\${LOG_FILE:=log\/graylog2-server.log}/LOG_FILE=\${LOG_FILE:=\/var\/log\/graylog2-server.log}/' /etc/init.d/graylog2

Next, install the startup script:

sudo update-rc.d graylog2 defaults

Now we can commence the Graylog2 server with the service command:

sudo service graylog2 start

The next stride is to install the Graylog2 web interface. Let's do that now!

Install Graylog2 Web interface

We will download and install the Graylog2 v.0.20.2 web interface in /opt with the following controls:

cd /opt; sudo wget https://github.com/Graylog2/graylog2-web-interface/releases/download/0.20.2/graylog2-web-interface-0.20.2.tgz
sudo tar xvf graylog2-web-interface-0.20.2.tgz

Let's create a symbolic link to the newly created directory, to simplify the directory name:

sudo ln -s graylog2-web-interface-0.20.2 graylog2-web-interface

Next, we want to configure the web interface's confidential important, the application.confidential parameter in graylog2-web-interface.conf. We will generate another important, as we did with the Graylog2 server configuration, and attach it with sed, like so:

SECRET=$(pwgen -s 96 1)
sudo -E sed -i -e 's/application\.secret=""/application\.secret="'$SECRET'"/' /opt/graylog2-web-interface/conf/graylog2-web-interface.conf

Now ajar the web interface configuration register, with this command:

sudo vi /opt/graylog2-web-interface/conf/graylog2-web-interface.conf

Now we need to modify the web interface's configuration to choose the graylog2-server.uris parameter. This is a comma been database of the server REST URIs. Since we only have one Graylog2 server node, the ideal should match that of rest_listen_uri in the Graylog2 server configuration (i.e. "http://127.0.0.1:12900/").

graylog2-server.uris="http://127.0.0.1:12900/"

The Graylog2 web interface is now configured. Let's commence it up to try-out it out:

sudo /opt/graylog2-web-interface-0.20.2/bin/graylog2-web-interface

You will know it began properly when you see the following two lines:

[info] play - Application started (Prod)
[info] play - Listening for HTTP on /0:0:0:0:0:0:0:0:9000

knocked CTRL-C to kill the web interface. Now let's install a startup script. You can either create your own, or download one that I created for this tutorial. To download the script to your environment directory, use this command:

cd ~; wget /images/article/graylog2_graylog_simple_setup_v2.png/graylog2-web

Next, you will want to copy it to /etc/init.d, and action its ownership to set and its permissions to 755:

sudo cp ~/graylog2-web /etc/init.d/
sudo chown set:set /etc/init.d/graylog2-web
sudo chmod 755 /etc/init.d/graylog2-web

Now you can install the web interface init script with this command:

sudo update-rc.d graylog2-web defaults

commence the Graylog2 web interface:

sudo service graylog2-web start

Now we can use the Graylog2 web interface. Let's do that now.

Configure Graylog2 to Receive syslog messages

Log into Graylog2 Web interface

In your best-loved browser, go to the port 9000 of your VPS's public IP addresses:

http://gl2_public_IP:9000/

You should see a login screen. Enter "admin" as your username and the password the admin password that you set earlier.

Once logged in, you will see something like the following:

Graylog2 Dashboard

The radiating chromatic "1" is a notification. If you depression on it, you will see a communication that says you have a node without any running inputs. Let's increase an input to collect syslog messages over UDP now.

Create Syslog UDP Input

To increase an input to collect syslog messages, depression on Inputs in the System menu on the right side.

Now, from the drop-down menu, appoint Syslog UDP and depression Launch brand-new input.

A "Launch a brand-new input Syslog UDP" window will sound up. Enter the following information:

  • Title: syslog
  • Port: 514
  • Bind addresses: gl2_private_IP

Then depression Launch.

You should now see an input labelled "syslog" in Running local inputs part (and it should have a chromatic blow that says "running" in it), like so:

Graylog syslog input

Now our Graylog2 server is prepared to collect syslog messages from your servers. Let's configure our servers to send their syslog messages to Graylog2 now.

Configure rsyslog to Send to Your Graylog2 server

On all of the servers that you want to send syslog messages to Graylog2, do the following stages.

Create a rsyslog configuration register in /etc/rsyslog.d. We will call ours 90-graylog2.conf:

sudo vi /etc/rsyslog.d/90-graylog2.conf

In this register, increase the following lines to configure rsyslog to send syslog messages to your Graylog2 server (replace gl2_private_IP with your Graylog2 server's independent IP addresses):

$template GRAYLOGRFC5424,"%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msg%\n"
*.* @gl2_private_IP:514;GRAYLOGRFC5424

Save and quit. This register will be loaded as part of your rsyslog configuration from now on. Now you need to restart rsyslog to put your action into effect.

sudo service rsyslog restart

After you are completed configuring rsyslog on all of the servers you want to observe, let's go back to the Graylog2 web interface.

Viewing Your Graylog2 Sources

In your best-loved browser, go to the port 9000 of your VPS's public IP addresses:

http://gl2_public_IP:9000/

depression on Sources in the top bar. You will see a database of all of the servers that you configured rsyslog on. Here is an instance of what it might look like:

Graylog2 Sources

The hostname of the sources is on the left, with the number of messages collected by Graylog2 on the right.

Searching Your Graylog2 Data

After letting your Graylog2 accumulate messages for some moment, you will be able to search through the messages. As an instance, let's search for "sshd" to see what category of SSH activity is happening on our servers. Here is a piece of our results:

Graylog2 Example Search

As you can see, our example search results revealed sshd logs for various servers, and a lot of failed set login attempts. Your results may vary, but it can help you to identify many issues, including how unauthorized users are attempting to access your servers.

In addition to the basic search practicality on all of your sources, you can search the logs of an accurate host, or in an accurate moment frame.

Searching through data in Graylog2 is helpful, for instance, if you would like to review the logs of a server or several servers after an incident has occurred. Centralized logging makes it simple to correlate related incidents because you do not need to log into aggregate servers to see all the events that have happened.

For more information on how the search bar works, check out the official documentation: The Search Bar informed

Conclusion

Now that you have Graylog2 set up, feel free to explore the other practicality that it offers. You can send other symbols of logs into Graylog2, and set up instruments (or reformat logs with app like logstash) to make the logs more structured and searchable. You can also look into diversifying your Graylog2 environment by separating the elements and increasing redundancy to increase performance and convenience.

Good fortune!

By Mitchell Anicas
Reference: digitalocean