Linux iTOps Tube

Wednesday, 23 November 2011

Speeding up MRTG with RRDtool

Speeding up MRTG with RRDtool

MRTG is a very useful program but it has a limitation. All the graphs and web pages are recreated each time a device is polled. This can potentially overload your MRTG server especially if you have a large number of monitored devices and the graphs take more than five minutes to generate. RRDtool is an application written by the creator of MRTG that can store general purpose data, but generates graphs on demand. Integrating MRTG with RRDtool can have very noticeable performance benefits. The example that follows will show you how to quickly implement a general purpose solution.


Scenario

The use of RRDtool is needed to reduce the load on a monitoring server that has been experiencing very sluggish performance due to the amount of MRTG graphs it has to regenerate every polling cycle.


  • Due to space constraints, the RRD database needs to be located in the /var partition.
  • The server has a default Apache configuration with the CGI files needed for dynamically generated content being located in the /var/www/cgi-bin directory.
  • A CGI script is required that will read the new MRTG data in RRDtool format.
  • The MRTG configuration file is /etc/mrtg/mrtg.cfg.

Here's how to proceed.


Installing RRDtool

The RRDtool and RRDtool PERL module file can be downloaded from its website athttp://people.ee.ethz.ch/~oetiker/webtools/rrdtool/, but installation can be tricky as the installation program may look for certain supporting libraries in the wrong directories.


Fortunately the prerequisite rrdtool and rrdtool-perl packages now come as part of most Linux distributions. For more details on installing packages, see Chapter 6, "Installing Linux Software").


Storing the MRTG Data in RRDtool Format

This phase of the integration process can be done in a few minutes, but the steps can be tricky:

  • The first step is to add some new options to your cfgmaker command. The first indicates that MRTG should only store rrdtool formatted data, and the second defines the /var/mrtg directory in which it should be stored. For added security, the directory should be external to your web server's document root.

--global 'LogFormat: rrdtool' --global "workdir: /var/mrtg" --global 'IconDir: /mrtg'

Finally, you should also specify an icon directory which specifies the location of all miscellaneous MRTG web page icons. The RRD web interface script we'll install later uses an incorrect location. The icon directory /mrtg is actually a partial URL location. In this Fedora scenario we are using the default Apache configuration which locates the MRTG icon files in the /var/www/mrtg directory. If you are using a non default Apache MRTG configuration or are using other Linux distributions or versions you may have to copy the icons to the custom directory in which the MRTG PNG format icon files are located.

The cfgmaker program is simple to use and is covered in in Chapter 22, "Monitoring Server Performance".

  • The next step is to create the data repository directory /var/mrtg and make it be owned by the apache user and process that runs the default Linux web server application.

[root@bigboy tmp]# mkdir /var/mrtg
[root@bigboy tmp]# chown apache /var/mrtg
[root@bigboy tmp]#

Note: If you are using SELinux you'll have to change the context of this directory to match that of the /var/www/html directory so that the apache process will be able to read the database files when your CGI script needs them. These commands compare the contexts of the both directories and apply the correct set to /var/mrtg.
Please refer to Chapter 20, " The Apache Web Server" for more details on file contexts with Apache.

[root@bigboy tmp]# ls -alZ /var/www | grep html
drwxr-xr-x  root     root     system_u:object_r:httpd_sys_content_t html
[root@bigboy tmp]# ls -alZ /var | grep mrtg
drwxr-xr-x  apache   root     root:object_r:var_t              mrtg
[root@bigboy tmp]# chcon -R -u system_u -r object_r -t httpd_sys_content_t /var/mrtg
[root@bigboy tmp]#

  • We now need to test that the RRD files are being created correctly. Run MRTG using the /etc/mrtg/mrtg.cfg file as the source configuration file then test to see if the contents of the /var/mrtg directory have changed. Success!

[root@bigboy tmp]# ls /var/mrtg/
localhost_192.168.1.100.rrd
[root@bigboy tmp]# 

The files are being created properly. Now we need to find a script to read the new data format and present it in a web format. This will be discussed next.

The MRTG / RRDtool Integration Script

The MRTG website recommends the script located on the mrtg-rrd website (http://www.fi.muni.cz/~kas/mrtg-rrd/) as being a good one to use. Let's go ahead and install it.

  • Download the script using wget. The site lists several versions; make sure you get the latest one.

           => `mrtg-rrd-0.7.tar.gz'
Resolving ftp.linux.cz... 147.251.48.205
Connecting to ftp.linux.cz|147.251.48.205|:21... connected.
Logging in as anonymous ... Logged in!
...
...
...
15:24:50 (53.53 KB/s) - `mrtg-rrd-0.7.tar.gz' saved [20863]
[root@bigboy tmp]# ls
mrtg-rrd-0.7.tar.gz
[root@bigboy tmp]#

  • Extract the contents of the tar file.

[root@bigboy tmp]# tar -xzvf mrtg-rrd-0.7.tar.gz 
mrtg-rrd-0.7/
mrtg-rrd-0.7/COPYING
mrtg-rrd-0.7/FAQ
mrtg-rrd-0.7/TODO
mrtg-rrd-0.7/Makefile
mrtg-rrd-0.7/mrtg-rrd.cgi
mrtg-rrd-0.7/ChangeLog
[root@bigboy tmp]#

  • Create the /var/www/cgi-bin/mrtg directory and copy the mrtg-rrd.cgi file to it.

[root@bigboy tmp]# mkdir -p /var/www/cgi-bin/mrtg
[root@bigboy tmp]# cp mrtg-rrd-0.7/mrtg-rrd.cgi /var/www/cgi-bin/mrtg/
[root@bigboy tmp]#

  • Edit the mrtg-rrd.cgi file and make it refer to the /etc/mrtg/mrtg.cfg file for its configuration details, or you can specify all the .cfg files in your /etc/mrtg directory.

#
# File: mrtg-rrd.cgi (Single File)
#
 
# EDIT THIS to reflect all your MRTG config files
BEGIN { @config_files = qw(/etc/mrtg/mrtg.cfg); }




#
# File: mrtg-rrd.cgi (multipl .cfg files)
#
 
# EDIT THIS to reflect all your MRTG config files
BEGIN { @config_files = </etc/mrtg/*.cfg>; }


  • You should now be able to access your MRTG RRD graphs by visiting this URL:

http://www.my-web-site.org/cgi-bin/mrtg/mrtg-rrd.cgi


Once installed, RRDtool operates transparently with MRTG. You'll have to remember to add the RRD statements to any new MRTG configurations and also add the configuration file to the CGI script. Our monitoring server can now breathe a little easier.

No comments:

Post a Comment