Novell Home

How to Install and Setup Munin on SLES 10

Novell Cool Solutions: Feature
By David Mair

Digg This - Slashdot This

Posted: 6 Oct 2006
 

Table of Contents:

Background:

Munin is a Linux host monitoring application written in Perl. It generates html output with graphs that illustrate the state over time for various host properties, e.g. amount and type of memory utilization, CPU utilization, interrupt and network activity and more. In fact, you can expand it to suit your needs.

What's cool about Munin is that it is very easy (and quick) to setup, it can monitor multiple hosts but requires only one point of observation, it's extensible, it requires very few system resources and, frankly, I like the style of the graphs. The memory status one gives a great view of trending behaviour in all memory allocation classes in one view.

It's like a lot of open source software, it's cool as much because it's the one you found and it has just as much functionality as the one the other guy found. But Munin is really powerful, it lets you look inside your systems and see how they change over time. You can spot trends and hot-spots at a glance.

Munin uses a client/server model to permit one host to generate the html for many hosts monitored. The monitoring is polling based so there is only any resource usage on the host during the poll event, which takes a few seconds in my testing. In my case the polling is configured to run every five minutes. The statistics themselves come from properties the host is already counting, mostly from proc filesystem contents. Munin just gathers them every five minutes and organizes them into graphs.

I came across Munin by accident several months ago but never had a use for it until recently. It's one of those things I bookmark under "Things to get back to". I was recently engaged at a customer site where there were unexplained crashes on a cluster system. We were already confident it was load related but didn't know the specifics. I thought that Munin was probably easy to set up and could tell us what was going on. I was able to setup a Munin server, a Munin client and an Apache server (the latter from source) to host Munin monitoring in less than an hour. After the next processing cycle we could see that the servers in the cluster needed more memory. We were so impressed that we left Munin on the servers to permit long term monitoring.

Basics:

These instructions were prepared by carrying out an Apache/Munin install on a SLES10 Server in VMWare Workstation 5.5.2. The C/C++ compilers and kernel source were added to the default install package set.

  • There are several elements that need to be downloaded, always verify downloads via checksums or digital signatures when they are available.
    • When using signatures you will probably have to get and import the public key for the signer. Be sure you get this only from the project site and not from a mirror — even if you get the other elements from a mirror.
  • I like to create a directory tree to organize the downloaded elements before installing them. For example, in this case I did the following:
    • mkdir -p /data/munin
    • I then downloaded all the files to that directory, expanded, built and installed them from there as necessary. Note that this isn't the directory the files were installed to, it's the workspace I used to perform the install from.

Apache Install:

If you already have an Apache server you want to use to publish Munin data then skip the remainder of this section completely.

  • Get the Apache source from http://httpd.apache.org/.
    • Use the download link on the left to download from a mirror site.
    • Be sure to verify the download with the PGP signature (or MD5 checksum) and verify the download is untainted and uncorrupted. The Apache web site has instructions at the bottom of the download page.
  • Expand the Apache source tarball.
  • cd into the directory that was created.
  • ./configure
  • If there are any errors, resolve them based on the error message. Usually errors are caused by missing dependencies (required libraries for example). The message should explain what was not found. The SLES install media should contain anything that was missing. Using the Software Management tool in YaST select the search view and just search for the missing element by name.
  • If you already have an Apache server and want to host Munin on a different instance of Apache then use: ./configure -prefix=<path you want to install to >
  • make
  • make install
  • This should put an Apache server in the default install directory (/usr/local/apache2 for Apache 2.2.3 for example) or in the directory you specified as a prefix.
  • You now need to configure Apache.

Apache Configuration:

This is not intended as a description of how to configure Apache in general. It is only a description of how to minimally configure an Apache instance to publish Munin.

  • Open the Apache configuration file (/usr/local/apache2/conf/httpd.conf in a default install of Apache 2.2.3)
  • Locate the DocumentRoot configuration statement (it will be at the start of a line and will not begin with a # character)
  • Note the directory parameter to the DocumentRoot statement (/usr/local/apache2/htdocs in a default install of Apache 2.2.3)
  • Create a sub-directory of the document root for Munin data, e.g.
    mkdir -p /usr/local/apache2/htdocs/munin
  • Change the owner:group to match that of the Apache DocumentRoot directory, e.g.
    chown -R wwwrun:daemon /usr/local/apache2/htdocs/munin
  • That's sufficient for a basic configuration. More complex configurations involving directory aliases or authentication are left as an exercise for the reader.
  • Start Apache:
    /usr/local/apache2/bin/apachectl start
  • If you want Apache to start at boot time, create a link to apachectl in /etc/init.d and enable it:
    ln -s /usr/local/apache2/bin/apachectl /etc/init.d/apache
    chkconfig apache on

Munin General:

There are two elements to Munin, the node and the server. The node is the host being monitored. The server hosts the statistical data and generates the html views for the host being monitored.

The Munin project is here: http://munin.projects.linpro.no/.

Find the "Download" section and click the "Download" link for the "Stable" version (1.2.4 as of this writing)

At the files page be sure to download both the Munin and munin-node rpms. You'll note that there is no SLES 10 version (as of writing). The SuSE Linux 10.0 versions and the SLES9 versions work well because Munin is all user mode code.

Installing a Munin Node:

A Munin node is the host being monitored. For these purposes, we'll use the same host for the Munin node as is being used for the Munin and web services but the method is identical for a different host.

The rpm we will be using is the one named in the form munin-node-version. If you try to install it now you will observe some of the following error messages:

error: Failed dependencies:
	perl-Net-Server is needed by munin-node-1.2.4-1.noarch
	sysstat is needed by munin-node-1.2.4-1.noarch

sysstat and perl-Net-Server are on the SLES10 install media:

  • Start YaST
  • Select the Software group
  • Click the Software Management icon
  • Type sysstat into the search field
  • Press the Search button
  • Check systat in the package list
  • Press the Accept button
  • Agree to any dependencies (probably just image drawing libraries)
  • When the package install completes click Yes in response to the question about installing more packages
  • Type perl-Net-Server into the search field
  • Press the Search button
  • Check perl-Net-Server in the package list
  • Press the Accept button
  • Agree to any dependencies (probably none)
  • When the package install completes, click No in response to the question about installing more packages
  • Close YaST

Use rpm to install munin-node, e.g.:
rpm -i munin-node-1.2.4-1.sl100.rpm

rpm should install the software, create the init.d service and enable it for runlevels 3 and 5 then start it up.

Configuring a Munin Node:

Stop munin-node:
/etc/init.d/munin-node stop

  • Edit /etc/munin/munin-node.conf
  • Find the host_name setting (it will probably be commented out with a #)
  • Remove any comments (#) from the start of the line
  • Replace the default hostname (lav) with a name you want to use for this monitored node (e.g. the hostname)
  • Munin listens on a tcp port, if you want to restrict access to the Munin node then add the IP address of the Munin server in an allow statement (there should be one for 127.0.0.1 already in the file). Note that the allow value is a regexp with start and end of line markers and dots have to be escaped, e.g.^10\.5\.20\.1$ to permit the host with IP address 10.5.20.1 to poll the munin-node
  • The things that Munin can monitor are expressed in perl scripts in /etc/munin/plugins/. If there are things you don't want to be monitored then add an ignore_file statement regular with a expression identifying the script to ignore. For example, I didn't find the hard disk temperature to be a useful statistic, so I added:

    ignore_file hddtemp_smartctl$.
  • Save the file and exit from the editor

Start munin-node:
/etc/init.d/munin-node start

Installing a Munin Server:

A Munin server is the host that will be polling Munin nodes, storing statistics and generating html views of those statistics.

The rpm we will be using is the munin-version. If you try to install the rpm at this point you will probably see the following error messages:

error: Failed dependencies:
	perl-HTML-Template is needed by munin-1.2.4-1.noarch

Unfortunately, this is not on the SLES10 install media and has to be installed manually via CPAN. CPAN is the Comprehensive Perl Archive Network, it hosts a huge library of Perl extensions:

  • Start the CPAN shell:
    perl -MCPAN -e shell
  • If this is the first time you have used CPAN then you will have to configure it.
  • If you observe any errors during the configuration it is probably because you have missing CPAN dependencies.
  • It's dependencies are mostly compiler/make related modules. I almost always add the C/C++ compiler packages when performing a Linux install, these are usually sufficient for CPAN.
  • You can ignore some of the missing download programs (lynx, nftpget, etc), as long as one download program exists CPAN should be happy.
  • At some point you will see a prompt to specify mirror servers to retrieve files from (it starts with a list of continents). Select download sources:
  • Select the relevant continent
  • Select the relevant country
  • You should now see a list of mirror servers. I don't bother with ftp sources so press Space Enter on each prompt until the list shows http servers. Around item 47 in my case. Note the number of the first http server.
  • I pick 10 servers so press Space Enter one more time to verify that there are 10 http servers following the first.
  • Enter your server selection as a list of server numbers separated by spaces, e.g.: 47 48 49 50 51 52 53 54 55 56.
  • Press Enter again to complete the selection.
  • You should now be at a CPAN shell prompt: cpan>
    Type:
    install HTML::Template
  • CPAN should retrieve, verify, build and install the Perl HTML-Template extension module
    Type:
    exit
  • At this point you have the Perl HTML::Template library installed as far as Perl is concerned but not as far as rpm is concerned. My workaround to this was to install the Munin server rpm with no dependency checking, e.g.:
    rpm -i --nodeps munin-1.2.4-1.sl1000.rpm

The Perl module for Munin, munin.pm, will actually be installed in a pre-programmed directory that assumes Perl 5.8.7 is installed. SLES10 includes Perl 5.8.8 so munin.pm must be moved (the following should be one line):

mv /usr/lib/perl5/vendor_perl/5.8.7/munin.pm /usr/lib/perl5/vendor_perl/5.8.8

Configure the Munin Service:

  • Open the Munin configuration file in an editor: /etc/munin/munin.conf
  • Modify the htmldir statement to reflect the directory you created earlier in your Apache DocumentRoot
  • Verify that the other directories referenced by munin.conf exist and create any that do not.
  • Set the owner and group to munin:munin for each directory except the htmldir
  • Set the owner OR the group (not both) to munin for the htmldir
  • Set the owner and group permissions for the htmldir to permit writing to the directory by both
  • In the munin.conf, find the sample host tree specification, it begins with a name in square brackets, e.g.: [lav]
  • Modify it to suit your munin-node
    • Change the name in the brackets to be the host name of the munin-node
    • Modify the address statement to specify the IP address or resolvable DNS host name for the munin-node
  • Create additional host tree sections for all munin-nodes you want this server to poll
  • Edit the crontab for the Munin user:
    crontab -u munin -e
    • Add a line to run munin-cron at a time interval of your choosing, e.g. For a poll of munin-nodes every minute:
      * * * * * munin-cron

      Every five minutes:

      */5 * * * * munin-cron

Viewing the Results:

Point your web browser at your Apache server and the DocumentRoot directory you created before, e.g.: http://myserver/munin/

Notes:

Most of the problems I've had with Munin have been caused by permissions on directories and files. Since the munin-node and server run as the munin:munin user:group failure to permit either or both of those write privileges can prevent Munin from recording any data. The Munin data gathering application is munin-cron but it doesn't like to be run as root unless forced and you are not going to see any permission errors as root. So, sudo as the Munin user to see any messages it generates: sudo -u munin munin-cron.

Munin is written in Perl and it runs per-class monitoring scripts from /etc/munin/plugins. You can add to these scripts, using the supplied ones as templates, and monitor other server state to suit yourself. The graphing is performed by the Munin service, you only have to provide the source of the data in a plugin. You should also be able to find additional scripts using your favorite Internet search engine.

Munin is an example of a host monitoring system. Others exist, Cacti for example.

Sample output

The output is actually in html pages that contain graphs, these are some samples of the graphs below. The system on which these were produced was mostly idle while they were recorded but note the ability to view trends directly and the min/max/average values on each graph. The memory usage one is particularly interesting given that it is a stacked bar graph so you can see the relationships between trending memory uses.

Note the per-day and per-week graphs. There are also per-month and per-year on each page. Note the per-device stats (though there is no activity on my CD-ROM drive, hdc). With each page there is a table describing each displayed statistic, e.g.:

This graph shows the I/O to and from block devices.

Field Internal name Type Warn Crit
hdc dev22_0_write derive None None I/O on device hdc
hda dev3_0_write derive None None I/O on device hda

Here's some more sample graphs (the per-day one only for each category):


Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com

© 2014 Novell