Check nrpe error receiving data from daemon

11 января 2013 г.

Nagios

NRPE

Мониторинг

FreeBSD

Debian

Linux

Вслед за статьей Вызов удаленных команд для Nagios с помощью NRPE решил записать решение на самую популярную проблему с NRPE, когда вы вызываете удаленную команду (в моем примере: плагин на FreeBSD, агент на Debian), то вам возвращается нечто такое CHECK_NRPE: Error receiving data from daemon

Эта ошибка, на деле, означает несколько проблем, а точнее их 5.

В Nagios Support FAQ есть замечательный топик Debugging «CHECK_NRPE: Error — Could not complete SSL handshake» errors объясняющий возможные проблемы, но не дающий конкретных решений.

Давайте разберем их:

Different versions. Make sure you are using the same version of the check_nrpe plugin and the NRPE daemon. Newer versions of NRPE are usually not backward compatible with older versions.

1

Если у вас установлены разные версии NRPE на серверах, то стоит для начала сделать их одинаковыми. Софт во FreeBSD-портах обычно свежее черем пакеты в Debian, поэтому доустанавливаем NRPE-агент в Debian вручную. Входим на страницу загрузки NRPE2 и качаем архив с NRPE (в моем случае это был nrpe-2.13):

wget -O nrpe-2.13.tar.gz http://downloads.sourceforge.net/project/nagios/nrpe-2.x/nrpe-2.13/nrpe-2.13.tar.gz
tar -xzf nrpe-2.13.tar.gz
cd nrpe-2.13

# Нам нужна поддержка SSL
sudo apt-get install libssl-dev

# Смотрим где храняться headers от libssl (в моем случае это "/usr/lib/i386-linux-gnu")
dpkg -L libssl-dev | grep libssl.a

# Собираем и устанавливаем
./configure --with-ssl=`which openssl` --with-ssl-lib=/usr/lib/i386-linux-gnu
make
sudo make install

# Останавливаем агента, подменяем бинарник, запускаем агента
sudo /etc/init.d/nagios-nrpe-server stop
sudo cp /usr/local/nagios/bin/nrpe /usr/sbin/nrpe
sudo /etc/init.d/nagios-nrpe-server start

SSL is disabled. Make sure both the NRPE daemon and the check_nrpe plugin were compiled with SSL support and that neither are being run without SSL support (using command line switches).
2

Это возникает, когда вы установили один из NRPE без поддержки SSL. В Debian он собирается по умолчанию с SSL, значит у вас проблема с портом во FreeBSD, при установке надо было выбрать поддержку SSL, теперь его переустановим:
cd /usr/ports/net-mgmt/nrpe2
make config
# Выбираем SSL
[*] SSL   Enable SSL support (disables plain-text server) 
make deinstall clean && make distclean && make install clean

Узнать собран ли NRPE-агент с поддержкой SSL или без, можно при запуске агента:
$ nrpe2 | head -n8

NRPE - Nagios Remote Plugin Executor
Copyright (c) 1999-2008 Ethan Galstad (nagios@nagios.org)
Version: 2.13
Last Modified: 11-11-2011
License: GPL v2 with exemptions (-l for more info)
TCP Wrappers Available

После переустановки порта (обратите внимание на предпоследнюю строку):
$ nrpe2 | head -n8

NRPE - Nagios Remote Plugin Executor
Copyright (c) 1999-2008 Ethan Galstad (nagios@nagios.org)
Version: 2.13
Last Modified: 11-11-2011
License: GPL v2 with exemptions (-l for more info)
>>> SSL/TLS Available: Anonymous DH Mode, OpenSSL 0.9.6 or higher required
TCP Wrappers Available

Incorrect file permissions. Make sure the NRPE config file (nrpe.cfg) is readable by the user (i.e. nagios) that executes the NRPE binary from inetd/xinetd.
3

Нехватает прав у пользователя nagios (так как везде, именно под ним запускается NRPE), проверяем:
# В Debian
sudo -u nagios /usr/lib/nagios/plugins/check_nrpe
# Во FreeBSD
sudo -u nagios /usr/local/libexec/nagios/check_nrpe2

Проверьте также права на конфигурационные файлы.
Pseudo-random device files are not readable. Greg Haygood noted the following… «After wringing my hair out and digging around with truss, I figured out the problem on my Solaris 8 boxen. The files /devices/pseudo/random* (linked through /dev/*random, and provided by Sun patch 112438) were not readable by the nagios user I use to launch NRPE. Making the character devices world-readable solved it.»



  


                       
  




4

Не встречался с такой проблемой, но для этого предусмотрен параметр в конфигурационном файле nrpe.cfg (надо его расскоментировать):
#allow_weak_random_seed=1

Unallowed address. If you’re running the NRPE daemon under xinetd, make sure that you have a line in the xinetd config file that say «only_from = xxx.xxx.xxx.xxx», where xxx.xxx.xxx.xxx is the IP address that you’re connected to the NRPE daemon from.
5

Скорее всего вы забыли указать allowed_hosts в конфигурационном файле nrpe.cfg и внешние хосты просто не уполномочены обращаться к NRPE-агенту, укажите нужные IP-адреса  через запятую и без пробелов:
allowed_hosts=127.0.0.1,203.0.113.41,203.0.113.42

Сам я под xinetd не запускал агента, меня устроил standalone-режим. Но именно 5-ая проблема самая распространенная в сети, но и сама простая.
Какие были у меня проблемы?
У меня самого было только две проблемы (разные версии NRPE и плагин без поддержки SSL). Вначале была ошибка:
/usr/local/libexec/nagios/check_nrpe2 -H <REMOTE_SERVER_ADDRESS> -c <COMMAND>
CHECK_NRPE: Error receiving data from daemon

А после того как я обновил (решение проблемы под номером 1) версии NRPE-агента (на Debian) до версии плагина (на FreeBSD), с которого запускал, то появилась следующая ошибка в /var/log/syslog (на Remote Host, Debian):
[nrpe] Error: Could not complete SSL handshake

Её я решил переустановив плагин на FreeBSD, как рассказывал в решении проблемы номер 2.
И совет напоследок, если не работает, начните с чтения лога /var/log/syslog или /var/log/messages, потому что nrpe будет писать именно туда:



  


                       
  




tail -f /var/log/syslog
tail -f /var/log/messages

Удачи!

NRPE — General Troubleshooting Tips

Problem Description

This KB article provides a troubleshooting methodology for NRPE problems.

The following KB article contains an explanation of how NRPE works and may need to be referenced to completely understand the problem and solution that is provided here:

When Troubleshooting NRPE issues, there is a general order of procedures for drilling down the problem. Start with the plugin itself, and then move to NRPE, and finally check your argument usage. If you follow the general steps below before dealing with support, your issue may be solved faster than expected as these are always the first steps a Nagios XI support representative will ask you to perform.

Test The Plugin Locally First

Log onto your remote server as root and and run the plugin (replace with your plugin:

If it does not work as expected, you may want to check the plugin’s usage as you may find some hints to why it is not working:

You may have to set some thresholds, usually warning ( -w ) and critical ( -c ) for a large number of plugins before they will work correctly. Once the plugin has been tested and working locally from the remote host, create a command directive for it in the nrpe.cfg file or adjust your existing command. Take a mental note of how you setup your arguments.

Verify That NRPE Is Working Locally And Open To Requests From The XI Server

On the remote host, run:

Or for systems not using xinetd:

If NRPE is not running, follow the steps in this KB article:

If NRPE is running, move on to testing the connection to the remote host from the XI server with check_nrpe . Log onto the Nagios XI server as root and run the following command, inserting the actual remote host IP address:

The command above should return the NRPE version of the remote host. If not, follow the steps in this KB article:

An important step in the previous command was to use the IP ADDRESS of the remote host. If your Nagios XI host object uses a DNS record, use that in the command above. It’s possible that the DNS record is incorrect.

Another similar mistake is when you use Core Configuration Manager to copy an existing host to provision a new host. The newly copied host still has the address of the host it was copied from (your forgot to change it). Your command line tests work OK because you are forced to type the address at the command line. A good troubleshooting technique is to make sure when you do your testing at the command line, make sure the arguments you type in match the values in the Nagios XI Core Configuration Manager objects.

Try The Full Command From The Command Line Interface On The XI Server

From the Nagios XI server command line interface, run the following command:

You will need to replace the remote host IP address and match your command and arguments to your command directives in your remote host nrpe.cfg .

If you do not get the expected output, check the plugin usage again to make sure your syntax is correct. As mentioned in the last step, a good troubleshooting technique is to make sure when you do your testing at the command line, make sure the arguments you type in match the values in the Nagios XI objects. Please refer to the following KB article for more information on this:

Setup The Service Check In XI

Create a new service for the check by navigating within the Nagios XI web interface Configure > Core Config Manager > Monitoring > Services > Add New.

Specify the Config Name and Description for the check.

Use check_nrpe in the Check command drop-down.

Next, set up the command arguments under Command view.

$ARG1$ is the remote command to be sent to the remote host through NRPE. This must match the command directive in the nrpe.cfg .

$ARG2$ is used for extra command arguments. Again, if you have defined any in the remote host’s nrpe.cfg this must match them.

The check needs to be applied to a host, so click the Manage Hosts button. Select a host from the list and click Add Selected. You should see the host appear in the right hand pane under Assigned. Now click Close.

Click the Check Settings tab. At minimum, we need to setup check intervals, attempts, and a period.

• Check interval specifies how often the check is run
• Retry interval specifies the time between check retries when the service check has failed (SOFT STATE)
• Max check attempts specifies the number of retries a check will attempt before it is marked as a HARD STATE fail
• The last required setting to set on this tab is the Check period
• This specifies what «time period» the check should run and can be configured for certain days and time frames. xi_timeperiod_24x7 will be fine for this example.

Last, click the Alert Settings and set the Notification period to «xi_timeperiod_24x7» , or to the time period of your choice. This specifies the time period for notifications. (emails, SMS, etc). Click Manage Contacts and add a contact to the check if you want.

Finally, click Save and Apply Configuration.

Alternatively you can run the NRPE or Linux Server configuration wizards which do all this for you.

Now when you navigate to Service Detail you will see your service check listed. It may take a minute for the service to change from pending to a STATE. From this page you can verify that your plugin is executing as expected.

For any support related questions please visit the Nagios Support Forums at:

Источник

check_nrpe error receiving data from daemon – How we fix it

by Ansu Anto | Apr 18, 2020

Are you receiving ‘check_nrpe: error receiving data from daemon’ in Nagios?

The reasons for this error are the bad configuration of NRPE, insecure connection with the remote server, etc.

At Bobcares, we monitor remote servers using Nagios, as part of our Server Management Services.

Today, let’s see how our Support Engineers fix this Nagios error.

How does Nagios monitor a remote machine?

Using Nagios is one of the preferred ways to monitor remote machines. Because it has flexible and configurable features.

The Nagios uses an add-on for monitoring a remote server, namely, Nagios Remote Plugin Executor aka NRPE. This add-on enables the execution of scripts to monitor remote machines.

The NRPE add-on has two parts, check_nrpe plugin in the Nagios server and NRPE daemon at the remote server.

Let’s see how it monitors.

• Initially, the Nagios server executes the check_nrpe plugin. This gives information to the remote server about the services to be checked.
• It contacts the NRPE daemon using an SSL connection.
• The NRPE daemon runs appropriate plugins to checks the services.
• The result is sent back to the check_nrpe plugin.
• Finally, the Nagios server displays the results.

Next, let’s see why does Nagios monitoring fails.

Why does Nagios fail to receive data from daemon?

When the check_nrpe plugin doesn’t receive data from the NRPE daemon, the Nagios servers shows the error,

A few reasons for this error are,

• Incorrectly compiled check_nrpe on the Nagios host.
• Using the no SSL switch (-n) with check_nrpe.
• Incorrect ownership and file permissions on the remote host.
• Firewall restrictions.

Let’s see how our Support Engineers fix this error.

How we fix check_nrpe error receiving data from daemon?

Now let’s see how we fix this error in different situations.

No SSL switch to connect with the remote host

Sometimes users add -n switch to the command definitions in the NRPE configuration file. This can end up in error if the remote host was compiled to connect with SSL.

We use the -n option only for test purposes if the SSL is not enabled on the remote end. In all other situations, we compile SSL in both machines. And we do not recommend the usage of -n switch in check commands.

Improper configuration

In some cases, improper configuration of /etc/nagios/nrpe.cfg file also can cause this error. Some major mistakes we often see are:

• Usage of white spaces in allowed_hosts. There should not be any white space in it.
• Similarly, the hosts must be separated by coma only.

We correct such errors in the config file.

Incorrect file permissions and ownership

Similarly, the permissions and ownership of Nagios files in the remote host also matter. The nagios user should have enough permissions on the related files and folders.

Hence we change ownership of folders in /usr/local/nagios using:

Additionally, the files must be readable by the Nagios user.

Firewall restrictions

If the firewall of the remote server is blocking the port used by Nagios, the same error can show up. So we also check the firewall rules in the server.

We always ensure to allow 5666 port in the server.

[Having trouble in monitoring your servers? – Our Experts are available 24/7.]

Conclusion

In short, check_nrpe error receiving data from daemon occurs mainly due to incorrectly compiled NRPE on the remote host. Today, we saw how our Support Engineers fixed this error.

PREVENT YOUR SERVER FROM CRASHING!

Never again lose customers to poor server speed! Let us help you.

Our server experts will monitor & maintain your server 24/7 so that it remains lightning fast and secure.

Источник

Check nrpe error receiving data from daemon

Commercial Support Clients: Clients with support contracts can get escalated support assistance by visiting Nagios Answer Hub. These forums are for community support services. Although we at Nagios try our best to help out on the forums here, we always give priority support to our support clients.

CHECK_NRPE: Error receiving data from daemon.

CHECK_NRPE: Error receiving data from daemon.

by tcicirello » Wed Feb 09, 2011 2:52 pm

This is probably a case of missing the obvious but i have tried everything i could think of and also what the nrpe.pdf doc suggests.

I am installing nrpe on centOS box using yum. The package installs without error.
Verion info: nagios-nrpe x86_64 2.12-1.el5.rf

I’ve added the check_nrpe command to /etc/nagios/objects/commands.cfg
# ‘check_nrpe’ command definition
define command <
command_name check_nrpe
command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
>

and the host IP address to nrpe.cfg

nrpe is configured to run as a daemon and shows as running:
ps auxw | grep nrpe
root 26257 0.0 0.0 61164 692 pts/3 S+ 13:43 0:00 grep nrpe
nagios 29589 0.0 0.0 39968 1084 ? Ss 13:18 0:00 nrpe -c /etc/nagios/nrpe.cfg -d

nrpe is listening on the correct port:
netstat -at | grep nrpe
tcp 0 0 mail.panoston.com:nrpe *:* LISTEN

The problem(s) arise when I run check_nrpe on the remote host.
Running without ssl yields
/usr/lib64/nagios/plugins/check_nrpe -H Remote IP address -n
CHECK_NRPE: Error receiving data from daemon.

Running with ssl yields:
/usr/lib64/nagios/plugins/check_nrpe -H 192.168.2.231
CHECK_NRPE: Error — Could not complete SSL handshake.

The only thing I haven’t tried is compiling from source using ./configure —enable-ssl. I assume yum will make the ssl option available on installation.

Any suggestions would be appreciated.

Re: CHECK_NRPE: Error receiving data from daemon.

by tonyyarusso » Wed Feb 09, 2011 3:29 pm

Re: CHECK_NRPE: Error receiving data from daemon.

by tcicirello » Wed Feb 09, 2011 4:33 pm

Re: CHECK_NRPE: Error receiving data from daemon.

by tonyyarusso » Wed Feb 09, 2011 5:53 pm

Is that ACCEPT listed before any REJECTS?

Do you have allowed_hosts set correctly in the NRPE config file?

Re: CHECK_NRPE: Error receiving data from daemon.

by tcicirello » Thu Feb 10, 2011 11:27 am

The only reject is the last line in the table — reject all.
The ouput is below.

iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
RH-Firewall-1-INPUT all — anywhere anywhere

Chain FORWARD (policy ACCEPT)
target prot opt source destination
RH-Firewall-1-INPUT all — anywhere anywhere
ACCEPT all — 192.168.21.0/24 anywhere

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

Chain RH-Firewall-1-INPUT (2 references)
target prot opt source destination
ACCEPT all — anywhere anywhere
ACCEPT icmp — anywhere anywhere icmp any
ACCEPT esp — anywhere anywhere
ACCEPT ah — anywhere anywhere
ACCEPT udp — anywhere 224.0.0.251 udp dpt:mdns
ACCEPT udp — anywhere anywhere udp dpt:ipp
ACCEPT tcp — anywhere anywhere tcp dpt:ipp
ACCEPT all — anywhere anywhere state RELATED,ESTABLISHED
ACCEPT all — 192.168.2.31 anywhere state NEW
ACCEPT all — IP Address anywhere state NEW
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:ssh
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:https
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:http
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:nrpe
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:ftp
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:netbios-ssn
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:mysql
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:smtp
ACCEPT udp — anywhere anywhere state NEW udp dpt:domain
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:domain
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:pop3
ACCEPT udp — anywhere anywhere state NEW udp dpt:ntp
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:imap
ACCEPT udp — anywhere anywhere udp spt:6277
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:responsenet
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:3121
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:vtr-emulator
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:openvpn
ACCEPT udp — anywhere anywhere state NEW udp dpt:openvpn
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:rsf-1
REJECT all — anywhere anywhere reject-with icmp-host-prohibited

The allowd_hosts line from nrpe.cfg

allowed_hosts= 96.37.142.40 (Monitor) 192.168.2.231 (Remote-for testing) 127.0.0.1

Источник

So I’m trying to set up a Nagios check_load service on a Red Hat server. I followed the documentation from Red Hat to install the NRPE client: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux_OpenStack_Platform/3/html/Installation_and_Configuration_Guide/Installing_and_Configuring_NRPE.html

On the Nagios GUI, I get this:

CHECK_NRPE: Error receiving data from daemon.

But when I log in to the Nagios server to debug, it looks like I can run this fine from the terminal:

root@portalmon:/Nagios# /usr/local/nagios/libexec/check_nrpe -H 10.0.XX.XXX -c check_load -t 30
OK - load average: 0.15, 0.10, 0.04|load1=0.150;15.000;30.000;0; load5=0.100;10.000;25.000;0; load15=0.040;5.000;20.000;0;

So I don’t think it’s a permissions problem. I think it’s some kind of path issue, but I can’t figure out why this is not working. Can anybody help?

Here is my service description:

define service{
  use                 dev-service
  host_name           [DEV] Luminis Admin DEV Portal
  service_description CPU-Load
  check_command       check_nrpe!"check_load"
}

I’m wondering if I’m even calling the same service when I run the terminal command above.

EDIT: I figured out the issue. Now working on the resolution. I changed Nagios to debug mode and by looking at /usr/local/nagios/var/nagios.debug I realized that the command that was being run was

/usr/local/nagios/libexec/check_nrpe -n -t 60 -H <hostname> -c check_load

instead of

/usr/local/nagios/libexec/check_nrpe -H 10.0.XX.XXX -c check_load -t 30

So something is appending the «-n» and it’s causing issues. The second command returns what I want, but the first returns

CHECK_NRPE: Error receiving data from daemon.

EDIT 2: Figured it out but I don’t have enough reputation to submit it as a valid answer, so below is what I typed.

I was able to resolve my issue. This is what I did.

1. In Nagios.cfg, I turned on debug mode and found the location of the debug file.
2. Replicated the issue while tailing the debug log. Found out the command the Nagios was executing was different than what I was expecting. It was appending the No SSL flag (-n).
3. In commands.cfg, I searched for check_nrpe. Found out that it was executing the command with -n. There was another command called check_secure_nrpe which runs check_nrpe without the -n flag.
4. Edited my service description to use check_secure_nrpe instead of check_nrpe.
5. Restarted Nagios.

The service now works as expected.

From DWIKI

dpkg-statoverride --update --add nagios www-data 2710 /var/lib/nagios2/rw
dpkg-statoverride --update --add nagios nagios 751 /var/lib/nagios2

       libssl.so.5 => /usr/lib/libssl.so.5 (0x80063a000)
       libcrypto.so.5 => /lib/libcrypto.so.5 (0x800784000)
       libwrap.so.5 => /usr/lib/libwrap.so.5 (0x800a16000)
       libc.so.7 => /lib/libc.so.7 (0x800b1f000)

       libcrypto.so.5 => /lib/libcrypto.so.5 (0x800784000)
       libc.so.7 => /lib/libc.so.7 (0x800b1f000)

NRPE — Nagios Remote Plugin Executor Copyright (c) 1999-2008 Ethan Galstad (nagios@nagios.org) Version: 2.15 Last Modified: 09-06-2013 License: GPL v2 with exemptions (-l for more info) SSL/TLS Available: Anonymous DH Mode, OpenSSL 0.9.6 or higher required TCP Wrappers Available

** POSSIBLE SECURITY RISK — COMMAND ARGUMENTS ARE SUPPORTED! ** ** Read the NRPE SECURITY file for more information **

Usage: nrpe [-n] -c <config_file> [-4|-6] <mode>

Options: -n = Do not use SSL <config_file> = Name of config file to use -4 = use ipv4 only -6 = use ipv6 only <mode> = One of the following operating modes: -i = Run as a service under inetd or xinetd -d = Run as a standalone daemon -d -s = Run as a subsystem under AIX

Notes: This program is designed to process requests from the check_nrpe plugin on the host(s) running Nagios. It can run as a service under inetd or xinetd (read the docs for info on this), or as a standalone daemon. Once a request is received from an authorized host, NRPE will execute the command/plugin (as defined in the config file) and return the plugin output and return code to the check_nrpe plugin.