• Heavy I/O on /var/log/nagios
• Heavy reads on /etc/nagios during starts and restarts.
• Bandwidth: consider connection bonding if you have more than one network interface. Nagios will use bandwidth, but you also need to consider the users connecting to the instance via a web browser.
• Memory: Nagios will run with 32 MB of memory. However, you need to consider Nagios is a web-based application and has a caching mechanism and Apache’s caching mechanism.
• Revision Control System (RCS): You want to be able to catalog changes to the production Nagios system.

Nagios Installation

1. Open yast2. Open: Software → Software Management
   • Install Packages:
     Apache2
     Apache2-prefork
     Apache2-mod_php5
     nagios
     nagios-nsca
     nagios-nsca-client
     nagios-plugins
     nagios-plugins-extras
     nagios-www
     rcs – if the package was not installed during the operating system installation.
   • Press: “Continue” to accept additional packages. You should see packages like fping and libmcrypt to be added to the RPM configurations.
   • After the packages have installed, select: “No” to “install or remove more packages?”
   • Quit out of yast2.
   • The installation is complete. At this time you can start Nagios (/etc/init.d/nagios start) and it will work. You can also start Apache (/etc/init.d/apache2 start) and it will work as well. However, you will still need to “tweak” Nagios and Apache to receive the desired results.

Where are the files?
In order to have your system become fully operational you need to know where all the pieces are located. Here is a listing of where the Nagios components are located:

Host, Service and Nagios Configurations: /etc/nagios/
Web Interface Files: /usr/share/nagios/
Nagios Log and Archives: /var/log/nagios/
Nagios Lock File: /var/run/nagios/
Nagios Log Rotating: /etc/logrotate.d/nagios
Nagios Data Files: /var/lib/nagios/
Nagios Plugins: /usr/lib/nagios/plugins/
CGI Files: /usr/lib/nagios/cgi/

Configuring Nagios and Web Interface

1. Add a nagcmd group to the system. sudo /usr/sbin/groupadd -g 5544 nagcmd
2. Add a nagios user. sudo /usr/sbin/useradd -u 1209 -g 5544 -c “Nagios” -d /home/nagios -s /bin/bash -m nagios.
3. You will also need to add the user nagios to the wwwrun group. Sudo /usr/sbin/usermod -g wwwrun nagios
4. Give the nagios user a password. sudo passwd nagios (write this down, because you will need to know it later).
5. Start Nagios: sudo /etc/init.d/nagios start.
6. In /etc/apache2/conf.d/nagios.conf, comment out the Directive lines “deny from all.”
7. Restart apache2: sudo /etc/init.d/apache2 restart.
8. Open a web browser and go to your host’s URL: http://your-host-name.com/nagios. You will be presented with the basic Nagios screen. However, if you select any of the “Monitoring” links you will receive a permissions error message.
9. Create a .htaccess_users file under /usr/share/nagios and add the nagiosadmin user and create a password: sudo htpasswd2 -c /usr/share/nagios/.htaccess_users nagiosadmin
10. Add the .htaccess file path to /etc/apache2/conf.d/nagios.conf under each “Directive” category:
    AuthName “Nagios Access”
    AuthType Basic
    AuthUserFile /usr/share/nagios/.htaccess_users
    Require valid-user
11. Restart apache: sudo /etc/init.d/apache2 restart
12. Uncomment the following lines in /etc/nagios/cgi.cfg:
    authorized_for_system_information
    authorized_for_configuration_information
    authorized_for_system_commands
    authorized_for_all_services
    authorized_for_all_hosts
    authorized_for_all_service_commands
    authorized_for_all_host_commands
13. Restart Nagios: /etc/init.d/nagios restart
14. Refresh your browser window and you will be prompted to login. Login using the nagiosadmin user name and the password you created in step 9.
15. SECURITY CONSIDERATIONS: Change the ownership of /usr/share/nagios/.htaccess_users to wwwrun (apache user) and the permissions to 400. This will ensure Apache can read the file. Root will always be able to read and append to the file. Restart apache for the change to take effect.

That is all there is to installing Nagios on SLES 10 SP3 using Novell’s Nagios bundle. I am sorry to say that Novell is bundling Nagios version 2.6, but it is a start.

Enjoy and please provide comments and or insight.

Our scenario involves two physical hosts (in my case I have created two virtual hosts using Xen). One host will house the master (node1) and the other the slave (node2). If you already have an instance of MySQL running in support of Nagios, this procedure will enable you to create a slave without interrupting your current installation and established permissions.

NOTE: Read through these instructions to make sure you understand them, because you will want to perform this procedure as quickly as possible. Nagios is going to be trying to writing to the master server while the instance is stopped during our data migration process.

1. Identify node1.
2. On node1 log into MySQL: sudo mysql (provide password).
3. Create a replication user: CREATE USER ‘repl’@'%.your domain.com’ IDENTIFIED BY ‘your password’; . Make sure you set up a separate account for the replication user, because the userid and password will be in plain text in your master.info and my.cnf files.
4. Grant the repl user rights: GRANT REPLICATION ON ‘*.*’ TO ‘repl’@'%.your domain.com’; The reason why we use the ‘*.*’ is to allow replication of all databases and hosts on the master instance. We use ‘%.your domain.com’ so we can create a slave anywhere on our networks.
5. Once that has been completed we need to proceed to node2 in order to modify the my.cnf file. If MySQL is running on the potential slave instance, stop MySQL.
6. Set the server-id = 2 (or the next number in your series of slaves).
7. Set the master-host = node1 hostname (I use the IP address of the master host).
8. Set the master-user = repl.
9. Set the master-password = your password (remember, I mentioned earlier that the userid and password for replication will be in plain text).
10. Set the log-bin = mysql-bin (you need to set this flag in order to have the ability to fail-over to this MySQL instance).
11. Now you want to start node2’s instance of MySQL. You will receive some warnings, but the instance should start and try to begin replication.
12. Log into node2’s instance of MySQL: sudo mysql and run: “SHOW SLAVE STATUS;”  You will see that the MySQL instance is now a slave.
13. Now, run “STOP SLAVE;” This will stop the slave on node2 in preparation for our next steps. You can quit the MySQL session on node2 and return to a command prompt.
14. On node1’s r instance of MySQL run: “FLUSH TABLES WITH READ LOCK;” This will prevent the tables from updating so we can obtain a snapshot.
15. Log into another session on node1 so we can perform a backup of the databases. From the command prompt run: mysqldump –all-databases –master-data > mydbdump.db. The “–master-data” flag will capture all our required log markers.
16. We will now restore the data from node1 to node2: sudo mysql < mydbdump.db. Remember, we froze node1, so we are restoring up-to-date data to node2.
17. Once the restore has completed, log into the MySQL instance on node2 and run: “START SLAVE;”
18. On node1 run: “UNLOCK TABLES;” This will begin the master’s data processing once more.

That’s all there is too it. I use a restore because I want my slave (node2) to spend as little time as possible updating or syncing to node1. This will also save on server load and network bandwidth. To test if replication is working properly perform this test:

• node1: Log into MySQL and run: “CREATE DATABASE testing;”, then “SHOW DATABASES;” and you should see the new database “testing”.
• node2: Log into MySQL and run: “SHOW DATABASES;” if replication is working you will now see the “testing” database.

I hope that helps with your MySQL instance on Nagios or Nagios XI. For more information on MySQL replication please visit: MySQL Replication .

Comments and or other solutions? You can always shoot me an email for questions at mikhail@ebusinessjuncture.com.

Using check_bigip_pool is also straight forward. You will need a hostname (hostname for F5), community (generally public), software version and pool name. The check_bigip_pool will return the number of nodes found running on the F5. So, you will need to work with the warning and critical percentages. HINT: If check_bigip_pool finds one of two nodes running and you only require one node running, set the warning to 49 and critical to 25. When both nodes are down you will receive a critical alert and notification.

Here are the configuration steps for this setup:

Define what is to be monitored (NOTE: this is just an outline, my organizational setup monitors 5 clients and 7 ports per client):

Client					URL				Port	HTTP Response
webstore.<your domain>.com	http://webstore			        80	 OK
webstore.<your domain>.com	http://webstore.com/WatchDog.tem 	80	 OK – I am here
f5host.<your domain>.com		your pool name			count of nodes up

*Service definition are placed within a template file. I prefer to place service definitions within a file that is descriptive of the function. You will also need to create a separate service definition for each service check, because you need to create distinct service_descriptions section. You can leave blanks in the service description; however, I do not because it will remove your ability to acknowledge problems via email. I also used the “hosts” flag instead of hostgroups, because I have 4 webstore servers, but not all of them are looking at the same URLs and ports. Also include a notes section, we will be using it later for our notification email.

Service Definition (nagios_host:/path-to-nagios/etc/objects/services/srv_check_http.cfg):

define service{
        use                     generic-service
        hosts                   hostname
        notes                   description of the service being monitored.
        service_description    	check_webstore
        check_command        	check_nrpe!check_webstore
        max_check_attempts      3
        normal_check_interval   5
        retry_check_interval    1
        check_period            24x7
        contact_groups          webstore_admins
        }

*Notice the variation in the check_command definition? This is so you can run the command
on the Nagios host and be listed under the defined host. 

F5 Service Definition (nagios_host:/path-to-nagios/etc/objects/services/f5_checks.cfg):

define service{
        use                      generic-service
        hosts                    f5host
        service_description      webstore_business_station
        notes                    There are 0 nodes online for http://business-station.coat.com.
        check_command            check_webstore_business_station_pool!
        max_check_attempts       3
        normal_check_interval    5
        retry_check_interval     1
        notification_period      24x7
        check_period             24x7
        contact_groups           webstore_admins
        }

*Host definition is placed within a template. I have created a separate template for each classification of hosts, which makes it easier to find and update hosts. The host definition would be the same for the f5 server.

Host Definition (nagios_host:/path_to_nagios/etc/objects/hosts/webstore_servers.cfg):

define host{
        use                      webstore-servers   ; Name of host template to use
        host_name                your host's name
        address                  ip_address
        alias                    short name for your host
        notes                    What does the host do <br>LOCATION:<CONSOLE: >
        icon_image               sun_logo.gif
        icon_image_alt           Sun Host
        hostgroups               Add your host groups for standard monitoring and specialty
                                 monitoring
        notification_period      24x7
        }

*Here is how I defined the commands. For separation purposes, I prefer to the definitions within a distinct configuration file and give each command name a distinct definition. This way if one port is not accessible, you can address the one port issue and not assume the entire web application is down.

Command Definition (nagios_host:/path_to_nagios/etc/objects/commands/cmd_http.cfg):

define command{
        command_name    	check_webstore
        command_line    	$USER1$/check_http  -H $ARG1$ -u $ARG2$ -p $ARG3$
        }

F5 Command Definition (nagios_host:/path_to_nagios/etc/objects/commands/cmd_check_bigip_pool.cfg):

define command{
        command_name    	check_webstore_pool
        command_line    	$USER1$/check_bigip_pool -H $HOSTADDRESS$ -C $ARG1$ -S $ARG2$ -P $ARG3$
                                 -w $ARG4$ -c $ARG5$
        }

*I combine the contactgroup with the contact definition in its own file. You may not have the luxury of email aliases, so this helps me manage the contacts for a specific host or service.

Contact Definition (nagios_host:/path_to_nagios/etc/objects/contacts/webstore_admins.cfg):

define contactgroup{
        contactgroup_name          webstore_admins
        alias                      WEBSTORE ADMINS
        members                    webstore_admins
        }
define contact{
        contact_name               webstore_admins
        alias                      WEBSTORE Admins
        contact_groups             webstore_admins
        host_notifications_enabled      0
        service_notifications_enabled  	1
        service_notification_period     24x7
        host_notification_period        24x7
        service_notification_options   	w,u,c,r
        host_notification_options      	d,r
        service_notification_commands   notify-webstore-service-by-email
        host_notification_commands     	notify-linux-host-by-email
        email                           webstore_support@your_domain.com
        can_submit_commands             1
        }

*Notifications are fun. I keep separate command files, because different groups need to receive different information. Notice that the service_notification_commands in the contacts matches the notification definition command name. Within the definition, you will find the notes section after the “-s” flag and also the method I use for enabling problem acknowledgments via a link within email.

Notification Definition (nagios_host:/path_to_nagios/etc/objects/commands/
webstore_admins_email_notification.cfg):

# Linux 'notify-webstore-service-by-email' command definition
define command{
        command_name    notify-webstore-service-by-email
        command_line    /usr/bin/printf "%b" "If this is a CRITICAL problem, please insure the
ecommerce on-call person is notified.\n\nHostname:\t\t$HOSTNAME$\nService:\t\t$SERVICEDESC$\n
State:\t\t\t$SERVICESTATE$\nDate/Time:\t\t$LONGDATETIME$\nAdditional Info:\t$SERVICEOUTPUT$
\nAcknowledge:\t\thttp://nagios_host.your_domain.com/nagios/cgi-bin/cmd.cgi?cmd_typ=34&host=
$HOSTNAME$&service=$SERVICEDESC$\nAuthor:\t\t$SERVICEACKAUTHOR$\nAcknowledgement:\t$SERVICEACKCOMMENT$"
| /usr/bin/mailx -s "$HOSTALIAS$:$SERVICENOTES$" $CONTACTEMAIL$ -r core.admin@coat.com
        }

Email Notification Message Received:

Subject: <Your service notes>
Body:
If this is a CRITICAL problem, please insure the ecommerce on-call person is notified.

Hostname:               webstore1.coat.com
Service:                check_webstore_8150
State:                  CRITICAL
Date/Time:              Wed Jan 6 02:24:39 EST 2010
Additional Info:        CRITICAL - Socket timeout after 10 seconds
Acknowledge:	        http://nagios_host.your_domain.com/nagios/cgi-bin/cmd.cgi?cmd_typ=34
                        &host=$service=service_name
Author:
Acknowledgement:

Hopefully this helps with your monitoring of web sites. Any questions please feel free to email me at mikhail@ebusinessjuncture.com. Here is a link to the document HTTP Monitoring Write-up

If anyone has comments or another method to perform this task, please leave a comment.

You can find information on SysNagios by visiting Nagios Exchange at: http://exchange.nagios.org/directory/Addons/Configuration/SysNagios/details

http://somethingdoug.com/thoughts/2009/10/05/nagiospluginoverhttp-0-11/

http://www.simonmclaughlin.co.uk/page/Android_Apps/

The video/audio quality isn’t the greatest, as we didn’t have ideal situations to work in. Sadly, our camcorder batteries died before we could complete the interview and the video is missing some really great discussions. We’ll catch up with Tobi in the future to bring you the rest. Tobi’s interview is spread across three videos.

We’re using a nice magic cron job that runs several Nagios plugins to ascertain the general health and well-being of the server. If enough of the performance of the server are out-of-bounds, the server will initiate an automatic restart. Not an ideal solution, but a reboot means the server will only be down for a minute instead of a few hours and the server should recover from memory exhaustion, etc. when its back up.

We’ll watch and see how this solution works over the next few days. The real solution will require deeper investigation, but its nice to have an elegant hack that gets 97% of the job done at 1% of the cost.

Q. Can you tell us a bit about yourself?
—
A. I am one of four Unix system administrators (engineers) at a large regional newspaper publishing company. I worked on Linux for the last ten years.
In 2005, I have specialized myself on the use of Nagios. In my very sparse freetime I enjoy motor cycling and use the machine to commute to work.

Q. Can you give us an overview of what your project (pnp4nagios) is?
—
A. PNP4Nagios (PNP) serves to process the performance data, prepared by the Nagios plugins. RRDTool is used for data visualization and storage.

Q. What are the primary advantages for a Nagios user to begin using pnp4nagios?
—
A. It’s very easy to install PNP and the requirements of maintenance work are minimal. In my opinion, trend graphs are one of the most important indicators to analyze problems.
A reliable alarm system like Nagios, paired with PNP, is a great assistance to get to the root of any problem.

Q. There are several graphing/trending addons available for Nagios. What inspired you to start work on pnp4nagios, and what are its advantages over other options?
—
A. The initial version was introduced in 2005, after the first german Nagios workshop. At this time, there were only a few similar Add-ons. The most popular of these was PerfParse. Though I didn’t like to manage data in MySQL databases and so I decided to publish my already existing scripts under www.nagios-portal.org. The synonym PNP (PNP is not PerfParse) emerged from this workshop.

Q. In your experience, do users require additional graphing/trending capabilities that are not already provided by pnp4nagios? For example, do they also use tools like Cacti?
—
A. Yes, I believe so. PNP is designed for the limited purpose to create trend graphs for Nagios. This is sufficient from the sysadmin’s point of view. However, managers require more exact methods of analysis. For example, easy comparison of several different rows of data. Cacti is a very good tool for data not contained in Nagios, and the zoom function of Cacti has been integrated into PNP. The developers of Cacti have done a really good Job.

Q. There are hundreds of community-contributed plugins out there. Are there plans to build a more extensive library or database of graphing templates to support nice layouts for these?
—
A. At first, all of these plugins need to show valid performance data. Sadly, too often are very little values attached to the developer guidelines, although a quite simple matter.
Of course I can visualize an area at exchange.nagios.org, where plugins are listed for which PNP templates already exists. I am alway happy to find plugins which offer appropriate templates, as in case of check_oracle_health, check_multi and check_mk. A template library is not planned for now.

Q. How long have you been working on pnp4nagios?
—
A. Since 2005. The project has developed strongly according to the needs of the users. I work diligently on PNP whenever time allows.

Q. You are the original author of pnp4nagios. Are there other active developers or contributors to the project?
—
A. Yes, thats right. But over time, many people have contributed to the development, and at this point I offer my gratefully thanks to them.
Hendrik Baecker and Wolfgang Nieder share work on the project. Hendrik is responsible for the parts written in C, Wolfgang takes care of the documentation and works untiringly on the mailing lists and on www.nagios-portal.org. Sven Velt looks after the Debain packages and Carlos de Nova the Spanish translation.

Q. How did you first come to know about Nagios and why did you decide to begin using it?
—
A. Its a long time ago. In 2003 we were using a commercial monitoring tool, and I was in charge of monitoring the SAP systems. However, the SAP agents of this tool never worked properly, so I installed Nagios and the SAP CCMS plugins. One year later, Nagios replaced the commercial tool. Since this time Nagios has played a central role in our organization.

Q. What do you see as being the most advantageous reasons for using Nagios?
—
A. Flexibility. Nagios is made for sysadmins. A good friend of mine said “Nagios can do everything you can do” and he was right.
Without Unix experience, the initial steps in Nagios are certainly not easy, but in the hands of an experienced sysadmin, Nagios is indeed a very powerful tool.

Q. Are there specific changes to Nagios that you’d like to see made in order to make pnp4nagios integration simpler?
—
A. Nagios allows plenty of headroom for integrating PNP. It would be advantageous to expand Nagios to facilitate the inclusion of sub configuration files, as in the Apache Web server.
This would simplify the task of including the appropriate config for packet maintainers. I would like to see automatic links to PNP in the future Webinterface without the need to define a notes or action URL.

Q. Are there any resources that you require in order to continue working on or improve the project?
—
A. I need more time! Developers, testers and translators are very welcome to assist. Simply register on the pnp4nagios-devel mailing lists, or send a direct mail to support@pnp4nagios.org

Q. What can the Nagios Community do to help improve pnp4nagios?
—
A. Respect the developer guidelines and write good plugins. Share your work on exchange.nagios.org and help each other on the mailing lists. Thats the way to keep a working community up and running.
I would be grateful to receive picture postcards if PNP pleases, and await them with interest to see in which countries PNP finds usage. My address on http://www.pnp4nagios.org/kontakt

Q. What does the future hold for pnp4nagios? What are your plans for it?
—
A.
PNP has a few limitations which we wish to eradicate. RRD databases are static, and therefor cannot update if the number of rows of data has changed.
The structure of databases cannot be changed at runtime. In future PNP will utilize separate RRD databases for each set of data. We are also developing the next version, the 0.6.x branch, under the PHP framework Kohana, also used by op5’s Ninja Project.