Install collectd statistics collecter on CentOS, RHEL, Ubuntu, Debian

Overview

collectd is a daemon which collects system performance statistics periodically and provides mechanisms to store the values in a variety of ways, for example in RRD files.

collectd gathers statistics about the system it is running on and stores this information. Those statistics can then be used to find current performance bottlenecks (i.e. performance analysis) and predict future system load (i.e. capacity planning). Source collectd.org

Install collectd statistics collecter on CentOS, RHEL, Ubuntu, Debian

Install some prerequistes

On CentOS/RHEL


# yum install rrdtool rrdtool-devel rrdtool-prel libgcrypt-devel gcc make gcc-c++

On Debian/Ubuntu


$ sudo su -
# apt-get install build-essential librrd2-dev libsensors4-dev libsnmp-dev libgcrypt-dev chkconfig

Note: Depending on the parameters you wish to monitor there may be some additional packages that needs to be installed. For a detailed list of all the parameter dependent packages read the README file under the collectd installer directory.

Download collectd latest tarball, at the time of writing the latest stable build is collectd-5.2.0. Untar it and run configure, make and make install.


# wget http://collectd.org/files/collectd-5.2.0.tar.gz
# tar zxvf collectd-5.2.0.tar.gz
# cd collectd-5.2.0
# ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var --libdir=/usr/lib --mandir=/usr/share/man --enable-all-plugins
# make
# make install

Create the intitialization script

On CentOS/RHEL


# cd collectd-5.2.0/contrib/redhat/
# cp init.d-collectd /etc/init.d/collectd

On Debian/Ubuntu


# vi /etc/init.d/collectd

Append the following to it


#! /bin/bash
#
# collectd - start and stop the statistics collection daemon
# http://collectd.org/
#
# Copyright (C) 2005-2006 Florian Forster
# Copyright (C) 2006-2009 Sebastian Harl
#

### BEGIN INIT INFO
# Provides: collectd
# Required-Start: $local_fs $remote_fs
# Required-Stop: $local_fs $remote_fs
# Should-Start: $network $named $syslog $time
# Should-Stop: $network $named $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: start the statistics collection daemon
### END INIT INFO

set -e

PATH=/sbin:/bin:/usr/sbin:/usr/bin

DISABLE=0

DESC="statistics collection and monitoring daemon"
NAME=collectd
DAEMON=/usr/sbin/collectd

CONFIGFILE=/etc/collectd.conf
PIDFILE=/var/run/collectd.pid

USE_COLLECTDMON=1
COLLECTDMON_DAEMON=/usr/sbin/collectdmon
COLLECTDMON_PIDFILE=/var/run/collectdmon.pid

MAXWAIT=30

# Gracefully exit if the package has been removed.
test -x $DAEMON || exit 0

if [ -r /etc/default/$NAME ]; then
. /etc/default/$NAME
fi

if test "$DISABLE" != 0 -a "$1" == "start"; then
echo "$NAME has been disabled - see /etc/default/$NAME."
exit 0
fi

if test ! -e "$CONFIGFILE" -a "$1" == "start"; then
echo "Not starting $NAME - no configuration ($CONFIGFILE) found."
exit 0
fi

if test "$ENABLE_COREFILES" == 1; then
ulimit -c unlimited
fi

if test "$USE_COLLECTDMON" == 1; then
_PIDFILE="$COLLECTDMON_PIDFILE"
else
_PIDFILE="$PIDFILE"
fi

check_config() {
if ! $DAEMON -t -C "$CONFIGFILE"; then
if test -n "$1"; then
echo "$1" >&2
fi
exit 1
fi
}

d_start() {
if test "$DISABLE" != 0; then
# we get here during restart
echo -n " - disabled by /etc/default/$NAME"
return 0
fi

if test ! -e "$CONFIGFILE"; then
# we get here during restart
echo -n " - no configuration ($CONFIGFILE) found."
return 0
fi

check_config

if test "$USE_COLLECTDMON" == 1; then
start-stop-daemon --start --quiet --oknodo --pidfile "$_PIDFILE" \
--exec $COLLECTDMON_DAEMON -- -P "$_PIDFILE" -- -C "$CONFIGFILE"
else
start-stop-daemon --start --quiet --oknodo --pidfile "$_PIDFILE" \
--exec $DAEMON -- -C "$CONFIGFILE" -P "$_PIDFILE"
fi
}

still_running_warning="
WARNING: $NAME might still be running.
In large setups it might take some time to write all pending data to
the disk. You can adjust the waiting time in /etc/default/collectd."

d_stop() {
PID=$( cat "$_PIDFILE" 2> /dev/null ) || true

start-stop-daemon --stop --quiet --oknodo --pidfile "$_PIDFILE"

sleep 1
if test -n "$PID" && kill -0 $PID 2> /dev/null; then
i=0
while kill -0 $PID 2> /dev/null; do
i=$(( $i + 2 ))
echo -n " ."

if test $i -gt $MAXWAIT; then
echo "$still_running_warning" >&2
return 1
fi

sleep 2
done
return 0
fi
}

d_status() {
PID=$( cat "$_PIDFILE" 2> /dev/null ) || true

if test -n "$PID" && kill -0 $PID 2> /dev/null; then
echo "collectd ($PID) is running."
exit 0
else
PID=$( pidof collectd ) || true

if test -n "$PID"; then
echo "collectd ($PID) is running."
exit 0
else
echo "collectd is stopped."
fi
fi
exit 1
}

case "$1" in
start)
echo -n "Starting $DESC: $NAME"
d_start
echo "."
;;
stop)
echo -n "Stopping $DESC: $NAME"
d_stop
echo "."
;;
status)
d_status
;;
restart|force-reload)
echo -n "Restarting $DESC: $NAME"
check_config "Not restarting collectd."
d_stop
sleep 1
d_start
echo "."
;;
*)
echo "Usage: $0 {start|stop|restart|force-reload|status}" >&2
exit 1
;;
esac

exit 0

# vim: syntax=sh noexpandtab sw=4 ts=4 :

Change permissions and start collectd


# chmod 755 /etc/init.d/collectd
# chown root:root /etc/init.d/collectd
# chkconfig --add collectd
# /etc/init.d/collectd start
# chkconfig collectd on

Open the collectd configuration file and make sure all basic plugins get loaded. You may see LoadPlugin directive followed by the parameter to monitor, it could have a # or ## sign in front. A single # means that the plugin is not active but is supported, just remove the # sign and it will start working. A double ## means that the plugin is not supported, this may be due to a missing package, in such senario just install the missing package and compile collectd again. You may refer the README file provided in the collectd installer directory to find what packages are required for a parameter to work.


# vi /etc/collectd.conf

Enable basic plugins


LoadPlugin network
LoadPlugin cpu
LoadPlugin memory

Also in a large environment you may consider creating a centralized collectd server and all clients sending data to collectd which then saves it to rrdtool. For clients to send data these additional plugins must be enabled on both client and server. Remove # sign in front of these plugins

Note either of syslog or logfile should be enabled.


# vi /etc/collectd.conf

Enable required plugins


LoadPlugin rrdtool
LoadPlugin network
LoadPlugin syslog
or
LoadPlugin logfile

Also append the following to server side collectd.conf. This will make collectd listen on the port 25827. Make sure you change the ip address below with the ip address of your collectd server


# Server
<Plugin "network">
Listen "192.168.209.161" "25827"
</Plugin>

Restart collectd


# /etc/init.d/collectd restart

Make sure collectd has started listening.


# netstat -tunlp | grep -i collectd

udp 0 0 192.168.209.161:25826 0.0.0.0:* 49475/collectd

Next hop over to the client and install collectd by following the process above. Once done enable the plugins you wish to monitor and the additional required onces to make it send data over the network.


# vi /etc/collectd.conf

Make sure the following plugins get loaded


LoadPlugin rrdtool
LoadPlugin network
LoadPlugin syslog
or
LoadPlugin logfile

Also append the following to client side collectd.conf


# Client
<Plugin "network">
Server "192.168.209.161" "25827"
</Plugin>

You can also encrypt the data and set authentication. For that lets create a auth file with two users and the passwords.


# mkdir -p /etc/collectd
# vi /etc/collectd/auth_file

Append the usernames and passwords.


debian01: password
centos01: password

On the servers side add SecurityLevel and AuthFile as shown below.


# Server
<Plugin "network">
<Listen "192.168.209.161">
SecurityLevel "Sign"
AuthFile "/etc/collectd/auth_file"
</Listen>
</Plugin>

On the client machine to set the client to encrypt the udp packets and provide username and password


# Client
Plugin "network">
<Server "192.168.209.161">
SecurityLevel "Encrypt"
Username "debian01"
Password "password"
</Server>
</Plugin>

Restart collectd


# /etc/init.d/collectd restart

On the server side you should now see a new directory named after the hostname of the client

# cd /var/lib/collectd/

So what we have for now is some data in rrd database but it is of no use unless we visualize this. There are some very nice tools available in market but I personally use Visage. If you are interested then below is a little screenshot of visage using collectd data.

visage6 300x176 Install collectd statistics collecter on CentOS, RHEL, Ubuntu, Debian

By the way you can check my next post which is about Visage here.