Log management using graylog2 0.11.0 on CentOS RHEL

Overview

Update: Please follow this link for the install guide of latest graylog2 version 0.12.0

Graylog has released the latest version of graylog server and web interface. This post will help you install both of these and assumes that you have elasticsearch installed and running, you can follow this post to install elasticsearch.

Log management using graylog2 0.11.0 on CentOS RHEL

Install some prerequisites


# yum install make gcc-c++ httpd httpd-devel readline-devel make httpd httpd-devel readline-devel gcc automake autoconf curl-devel openssl-devel zlib-devel apr-devel apr-util-devel sqlite-devel java git

The new version of graylog web interface require ruby version >= 1.9


# wget http://pyyaml.org/download/libyaml/yaml-0.1.4.tar.gz
# tar zxvf yaml-0.1.4.tar.gz
# cd yaml-0.1.4
# ./configure --prefix=/usr/local
# make
# make install
# wget http://ftp.ruby-lang.org/pub/ruby/1.9/ruby-1.9.3-p0.tar.gz
# tar xzvf ruby-1.9.3-p0.tar.gz
# cd ruby-1.9.3-p0
# ./configure --prefix=/usr/local --enable-shared --disable-install-doc --with-opt-dir=/usr/local/lib
# make
# make install

Verify the version

# ruby -v
ruby 1.9.3p0 (2011-10-30 revision 33570) [x86_64-linux]
# gem --version
1.8.11

Add 10gen Repository for mongodb


# vi /etc/yum.repos.d/mongodb.repo

Append the following to it
For x86_64 bit CentOS/RHEL


[10gen]
name=10gen Repository
baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64
gpgcheck=0

For i386 32 bit CentOS/RHEL


[10gen]
name=10gen Repository
baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/i686
gpgcheck=0

Install mongodb and other dependencies


# yum install mongo-10gen-server mongo-10gen readline-devel

Open the mongod.conf file in an editor of choice


# vi /etc/mongod.conf

Enable authentication by uncommenting the line


auth = true

Start mongodb


# /etc/init.d/mongod start
# chkconfig mongod on

Login to mongo db and add user and database for graylog2


# mongo
>use admin
>db.addUser('admin', 'password')
>db.auth('admin', 'password')
>use graylog2
>db.addUser('grayloguser', 'gray123')
>exit

Download the latest version of graylog2-server and untar it


# wget http://download.graylog2.org/graylog2-server/graylog2-server-0.11.0.tar.gz
# tar xvfz graylog2-server-0.11.0.tar.gz
# cd graylog2-server-0.11.0

Copy the conf files


# cp graylog2.conf.example /etc/graylog2.conf
# cp elasticsearch.yml.example /etc/graylog2-elasticsearch.yml
# mv graylog2-server-0.11.0 /opt/graylog2-server

Open the graylog2.conf file in an editor of choice


# vi /etc/graylog2.conf

Set syslog_enable_tcp = true line 12 and set mongodb username and password line 87.


syslog_enable_tcp = true
mongodb_user = grayloguser
mongodb_password = gray123

Create the initialization script


# vi /etc/init.d/graylog2-server

Append the following to it


#!/bin/bash
# graylog2-server: graylog2 message collector
# chkconfig: - 98 02
# description: This daemon start graylog2-server
# Source function library.
. /etc/rc.d/init.d/functions

CMD=$1
NOHUP=`which nohup`

STOP_TIMEOUT=30
BINARY=java
PROG=graylog2-server

HOME_DIR=/opt/graylog2-server
LOG_FILE=${HOME_DIR}/log/${PROG}.log
JAR_FILE=graylog2-server.jar
GRAYLOG2_CONFIG_SH=${GRAYLOG2CTL_DIR}/bin/graylog2_config.sh
CONF_FILE=/etc/graylog2.conf
PID_FILE=/var/run/graylog2.pid

[ -f $GRAYLOG2_CONFIG_SH ] && . $GRAYLOG2_CONFIG_SH

start() {
graylog2_status > /dev/null 2>&1
if [ ${RETVAL} -eq 3 ]
then
echo "Starting ${PROG} ..."
cd ${HOME_DIR}
$NOHUP > /dev/null 2>&1 ${BINARY} -jar ${JAR_FILE} -f ${CONF_FILE} -p ${PID_FILE} >> ${LOG_FILE} &
RETVAL=0
else
echo "${PROG} is already running"
fi
}
stop() {
echo -n $"Stopping $PROG: "
killproc -p ${PID_FILE} -d ${STOP_TIMEOUT} ${PROG}
RETVAL=$?
echo
[ $RETVAL = 0 ] && rm -f ${PID_FILE}
}

graylog2_status() {
status -p ${PID_FILE} ${PROG}
RETVAL=$?
}

restart() {
echo "Restarting ${PROG} ..."
stop
start
}
case "$CMD" in
start)
start
;;
stop)
stop
;;
restart)
restart
;;
status)
graylog2_status
;;
*)
echo "Usage $0 {start|stop|restart|status}"
RETVAL=1
esac

exit ${RETVAL}

Add it to chkconfig and start the service


# chmod +x /etc/init.d/graylog2-server
# chkconfig --add graylog2-server
# chkconfig graylog2-server on
# service graylog2-server start

Once you start graylog server, it will start listening on port 514, make sure that no other service is using port 514 on the machine, you would like to stop rsyslog if its running and using port 514. Also make sure you have configured rsyslog clients to send logs to this centralized server. To know more about centralized logging using rsyslog click here.

Download and install graylog2 web interface


# wget http://download.graylog2.org/graylog2-web-interface/graylog2-web-interface-0.11.0.tar.gz
# tar zxvf graylog2-web-interface-0.11.0.tar.gz
# mv graylog2-web-interface-0.11.0 /var/www/html/graylog2
# cd /var/www/html/graylog2/
# gem update
# gem install git rake bundler
# bundle install

Update the mongoid,yml file


# cd /var/www/html/graylog2/config
# vi mongoid.yml

Add the username and password


production:
host: localhost
port: 27017
username: grayloguser
password: gray123
database: graylog2

Create the indexes


# cd /var/www/html/graylog2
# bundle exec rake db:mongoid:create_indexes RAILS_ENV=production --trace

Install apache passenger module


# passenger-install-apache2-module

passenger01 300x176 Log management using graylog2 0.11.0 on CentOS RHEL

Create a new conf file to load passenger module


# vi /etc/httpd/conf.d/passenger.conf

Append the following to the file as instructed by your apache passenger module installer(Refer the screenshot above)


LoadModule passenger_module /usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.18/ext/apache2/mod_passenger.so
PassengerRoot /usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.18
PassengerRuby /usr/local/bin/ruby

Create virtual hosts


# vi /etc/httpd/conf/httpd.conf

Append the following code to the file


<VirtualHost *:80>
ServerName graylog.linuxdrops.com
DocumentRoot /var/www/html/graylog2/public
RailsEnv production
ServerAlias gray.logger
ErrorLog logs/graylog2-error_log
CustomLog logs/graylog2-access_log custom
<Directory /var/www/html/graylog2/public>
Allow from all
Options -MultiViews
</Directory>
</VirtualHost>

Restart apache


# /etc/init.d/httpd restart

Point your browser to

http://ipaddress-or-domainname

graylog02 300x176 Log management using graylog2 0.11.0 on CentOS RHEL

graylog011 300x176 Log management using graylog2 0.11.0 on CentOS RHEL