I feel one reason for the success of Nagios is the ability to integrate multiple monitoring schemes. If users want to monitor the environment actively, they can perform that task directly from the Nagios host or using NRPE, NSCA and or NSCLIENT++. My organization uses a combination of all the mentioned.

Notifications are simple to configure and using NDOUTILS and Nagvis you can set up excellent visual system diagrams. My organization actually purchased 4 42″ flat-screen televisions to create a control room consisting of Nagvis charts.

Keep up the excellent work Ethan! You and your team are putting forth an excellent open source project with Nagios. I am sure you will start to experience increased success with your current business model.

Mike Kniaziewicz, MIS

A couple of applications that I can name immediately would be a HRIS and or EPR system. You do need to establish parent/child relationships in order to quickly identify and resolve the issue.

Your concern about the parent/child logic is excellent. My research indicates that the relationship is OR dependent upon how the parent/child relationships are established. Let me explain with an example.

I work for a retail organization and we monitor all our stores from a central location (which will be converted to distributed monitoring in the next few months). Every store has an internal router providing gateway services. Corporate also has a couple of routers; one talks to the even numbered stores and the other router odd numbered stores. Between corporate and the stores is our ISP.

Servers within the stores have the store router set as their parent. When the entire router is down, the store servers report as being UNREACHABLE. However, if only the frame is down and the modem is active at the store, the server will report accurately. As long as Nagios can find a path to the object the check will be performed, other wise it will create notifications for the parent item and list downstream items as UNREACHABLE.

Multiple parent objects can be confusing at times. If you have a single application that depends upon applications running on two host you can list multiple hosts for the application. This is where service dependencies and host dependencies come into play. For the services you would have to create a service group, then use the service group name for your service dependency.

Can this get complex? Yes! That is why I would recommend keeping documentation from the start. I also use DIA to create diagrams, since a picture says a thousand words. I also place the process diagram in Nagvis and create the service checks in Nagios for the diagram. Whether I wanted it or not, I have built my Nagios system to the point I am the only one that can create modifications. Not because of the lack of documentation, but due to the complex system to monitor our infrastructure.

Hope this helps,
Mike Kniaziewicz, MIS

I’ve been digging the whole “parent/child” + “host/service dependencies” thing recently, since I’m rebuilding my nagios installation from scratch (major overhaul needed .

So far, I’ve been trying to carefully design the parent-child relationship between hosts, which allows for a much more precise identification of host states following the “reachability logic” that nagios cleverly applies.
Anyway, I found that some information in the official doc to be far from precise. Namely, when you have to define multiple parents for a host (think for example a typical scenario with a host with a clustered gateway – which means that there will be an outage when BOTH the gateways are down, for whatever reason). It’s not clear here if when defining multiple parents for a host, they will be treated with an AND or OR logic.. and I still have to make some tests in my lab environment for this.

Hosts and Services dependencies are a different beast (and, btw, not so “easy” nor “required”.. ). The idea here is slightly different from the parent-child relationships (whether they’re service or host depnendencies): parent-child relationships help/modify the reachability logic (thus providing more precision when distinguishing between a DOWN and UNREACHABLE state), while service/host dependencies are used to improve the notification schema (e.g., avoiding notifications for services/hosts that are brought down from an outage on one (or more) master service/host.

While we can all agree that reducing notifications to a minimum (in the quest for the root cause of problems) is a desired behaviour, it’s far more complicated than defining the parent/child relationship mentioned before.

I ended up using this rule for defining host/service dependencies: a service/host is dependant on another service/host, in nagios lingo, if the “master” service/host is VITAL for the “child” service/host to function/survive.

While it may sound logical, I found myself falling many times in the “everything depends from everything scenario”. For example, I’ve been monitoring a bunch of web-app servers using the usual plugins (check_http, and the like), and an oracle rdmbs (with check_oracle and the like). Since those web-app servers hosted an app that relies on the oracle db, it seemed logical at first to define a service dependencies from the master (oracle db) to the child (web-app server). Wrong! From the nagios standpoint, a fault in the oracle db won’t directly impact the web-app server instance, so it’s pointless to define that dependency, as it would suppress useful notifications that could arise at some point (both the db and the web-app truly down…). One could argue that in this case I could define a deeper check that doesn’t validate the state of the web-app server, but the app itself (maybe by scripting the logon process on the app, etc.), but that’s another story, since I’m in charge of running&monitoring the underlying infrastructure, not the actual “end user service”.

So, I ended up defining dependencies for HARD relationships, such as this one: I’m running an IMAP server that relies on a LDAP server for reaching user mailboxes and authenticating them: in this case, if the LDAP server dies, the IMAP server doesn’t work anymore. In this case a service dependency is welcome, as it suppresses notifications for the IMAP server being in a CRITICAL state (since the “master” LDAP service is the one to be blamed for the outage).

To further complicate things, I’m still confused about the “multi-parent” dependency mapping (e.g. when a service/host depends on multiple hosts/services). Again, the official docs don’t explain if a OR or AND logic (or even NO logic) is applied when evaluating the dependencies tree. What if I have a VIP on a loadbalancer, that depends on the availability of 5 web servers? (at least 1 of them should be available, so the OR logic should apply here…)

I still have faith that parent/child relationship and host/service dependencies can help a lot when defining the monitoring topology of your infrastructure, but they should always be cleverly thought and planned (btw, a wild use of dependencies causes lots of headaches in terms of configuration management, while the parent/child relationship generally doesn’t, at least imho).

Do you have some info about the dependency logic (AND/OR/WHATEVER) I mentioned above?

Drakpz

Thanks for the article. I had never heard of check_mk, the idea intrigues me. We are a Windows shop using Nagios with NRPE and NSClient++. We have done this for sometime with good success, however, I love the idea of a more prepackaged option that does NOT rely on other Windows components such as .net, wmi, etc. We will check it out, this may be a good data collector compliment/replacement with what we are using now.

Thanks again for the info,

-greg

1. During the analysis phase you would determine why and what you want to deploy in regard to Nagios.

2. The development phase would include modifications to the “spec” file for your environment. An example would be how my organization does not deploy Nagios down the /usr/home/nagios prefix, but under /opt/nagios. This would be changed inside the configuration file like so: “%define prefix /opt/nagios.”

3. Implementation would require either a manual deployment of the RPMs or you can use a system like we have in ZenWorks

4. Maintenance phase is a breeze when you consider that a RPM can be given an upgrade flag and that you should be able to build the rpm from the original spec file.

Neither I nor my organization would be interested in any form of central registration process. Especially when you consider that not all enterprise have the same requirements. I am in favor of the small Nagios core development team, but my organization needs the ability to deploy components that we require to our specifications.

In regard to Windows? I do feel that a bulk of Window’s user are more that likely using a product like IBM’s Tivoli. However, I am certain that the Window’s community could create an installation package for Nagios and its components. I think what stands in the way is the lack of revenue for the Window’s people. How many decades did it take before Apache was configured to actually work on Windows? On the bright side, we have 100+ servers running NSClient++ and NRPE, because our administrators took the time to make it work.

Compiling RPMs is not hard, if you know the secrets of what to look for within the spec file and make modifications. I spent two days building RPMs for RedHat 9 (shrike). We have legacy systems that we are upgrading, but need to monitor now. The RPM builds included nagios-plugins, nrpe and nsca. The difficulty here was the outdated RPM system and having to make “tweaks” to the configuration file as well as adding functionality like a nagios crontab to run the NSCA checks on the local system.

I feel RPM creation is a very important aspect of the SDLC, so that you do not have all the headaches of modifying configuration files every time you install Nagios or one of its components. Change management is important and every spec file and binary package should be checked into a revision control system, so changes can be recorded.

Thanks for your comments and I found them to be very engaging,
Mike Kniaziewicz, MIS

First, more than SDLC, admins are involved with change management, and specifically with configuration management. SDLC is good, but the fact is (or should be) that as admins we should worry about the config management process (with regard to software packages, etc).

BTW, I’ve been using nagios/plugins/nrpe custom built RPMs (Dag Wieers repo rules ), but at the end of the day, what kills me it’s not the complexity of distributing the packages, or tracking their dependencies, or seamlessly manage upgrades: I’m still stuck with some custom-made configuration management process, that often involves manual editing of config files, and NRPE is one of those beasts here.

As someone already suggested, it would be overkill to get a central “registration process” where nrpe and friends automagically download their conf and keep in sync with the main nagios conf. And that would very much adhere not with a SDLC framework, but with a Change Management one.

Also, official RPMs would indeed solve some maintenance headache on the linux side, but it’s not all linux out there, so… once again, I see wider margins of improvement in the config management processes.

MY 2 cents

I created this type of read-only for my organization’s Vice Presidents. I feel a very important aspect needs to be considered here, security. I recommend changing the htaccess.users file to a “dot” file. Changing the permissions to 400 and making the owner whatever Apache runs as. Root can modify any file and only Apache needs to authenticate against the file.

Remember, placing a .htaccess file within a directory covers the directory the file is in and any sub directories.

Just a little addition.

Hopefully this will clarify some problems users were having with the check_cluster command. The check_cluster command is great for checking all types of clusters. However, I have found the online documentation (gathered by a Google search) to only contain the same information.

Once again thanks for the clarification and hopefully members of the community benefit from this exchange.