Description
Hello!
Got this crash on the poller part of the service. After having runned librenms-service.py for little more than one hour.
Jan 11 10:36:41 srv-net101 librenms-service.py[1348043]: Poller_0-2(INFO):Completed poller run for 161 in 4.58s
Jan 11 10:36:41 srv-net101 librenms-service.py[1348043]: Traceback (most recent call last):
Jan 11 10:36:41 srv-net101 librenms-service.py[1348043]: File "/opt/librenms/librenms-service.py", line 48, in
Jan 11 10:36:41 srv-net101 librenms-service.py[1348043]: service.start()
Jan 11 10:36:41 srv-net101 librenms-service.py[1348043]: File "/opt/librenms/LibreNMS/service.py", line 404, in start
Jan 11 10:36:41 srv-net101 librenms-service.py[1348043]: sleep(self.config.master_resolution)
Jan 11 10:36:41 srv-net101 librenms-service.py[1348043]: File "/opt/librenms/LibreNMS/service.py", line 550, in reap
Jan 11 10:36:41 srv-net101 librenms-service.py[1348043]: handler = signal(SIGCHLD, SIG_DFL)
Jan 11 10:36:41 srv-net101 librenms-service.py[1348043]: File "/usr/lib/python3.8/signal.py", line 48, in signal
Jan 11 10:36:41 srv-net101 librenms-service.py[1348043]: return _int_to_enum(handler, Handlers)
Jan 11 10:36:41 srv-net101 librenms-service.py[1348043]: TypeError: 'int' object is not callable
Jan 11 10:36:42 srv-net101 librenms-service.py[1348043]: Poller_0-5(INFO):Completed poller run for 117 in 4.12s
Jan 11 10:36:42 srv-net101 librenms-service.py[1348043]: Poller_0-18(INFO):Completed poller run for 169 in 6.33s
Jan 11 10:36:42 srv-net101 librenms-service.py[1348043]: Poller_0-12(INFO):Completed poller run for 385 in 4.30s
Jan 11 10:36:42 srv-net101 librenms-service.py[1348043]: Poller_0-8(INFO):Completed poller run for 100 in 4.49s
Jan 11 10:36:43 srv-net101 librenms-service.py[1348043]: Poller_0-13(INFO):Completed poller run for 298 in 4.23s
Jan 11 10:36:43 srv-net101 librenms-service.py[1348043]: Poller_0-1(INFO):Completed poller run for 433 in 3.69s
Jan 11 10:36:44 srv-net101 librenms-service.py[1348043]: Poller_0-19(INFO):Completed poller run for 65 in 4.09s
Jan 11 10:36:44 srv-net101 librenms-service.py[1348043]: Poller_0-3(INFO):Completed poller run for 601 in 4.98s
Jan 11 10:36:45 srv-net101 librenms-service.py[1348043]: Poller_0-9(INFO):Completed poller run for 603 in 5.37s
Jan 11 10:36:51 srv-net101 librenms-service.py[1348043]: Poller_0-11(WARNING):Polling device 134 unreachable, waiting 60s for retry
Jan 11 10:36:51 srv-net101 librenms-service.py[1348043]: Poller_0-11(INFO):Completed poller run for 134 in 13.53s
Jan 11 10:37:00 srv-net101 librenms-service.py[1348043]: Ping_0-1(INFO):Running fast ping
Jan 11 10:37:00 srv-net101 librenms-service.py[1348043]: Billing(INFO):Calculating billing
Jan 11 10:37:00 srv-net101 librenms-service.py[1348043]: Alerting(INFO):Checking alerts
Jan 11 10:37:01 srv-net101 librenms-service.py[1348043]: Billing(INFO):Completed billing run for calculate in 0.08s
Jan 11 10:37:01 srv-net101 librenms-service.py[1348043]: Alerting(INFO):Completed alerting run for alerts in 0.12s
Jan 11 10:37:13 srv-net101 librenms-service.py[1348043]: Ping_0-1(INFO):Completed ping run for in 12.93s
Jan 11 10:38:00 srv-net101 librenms-service.py[1348043]: Ping_0-1(INFO):Running fast ping
Jan 11 10:38:00 srv-net101 librenms-service.py[1348043]: Billing(INFO):Calculating billing
Jan 11 10:38:00 srv-net101 librenms-service.py[1348043]: Alerting(INFO):Checking alerts
Jan 11 10:38:01 srv-net101 librenms-service.py[1348043]: Billing(INFO):Completed billing run for calculate in 0.09s
Jan 11 10:38:01 srv-net101 librenms-service.py[1348043]: Alerting(INFO):Completed alerting run for alerts in 0.12s
Jan 11 10:38:13 srv-net101 librenms-service.py[1348043]: Ping_0-1(INFO):Completed ping run for in 13.03s
Jan 11 10:39:00 srv-net101 librenms-service.py[1348043]: Ping_0-1(INFO):Running fast ping
Jan 11 10:39:00 srv-net101 librenms-service.py[1348043]: Billing(INFO):Calculating billing
Jan 11 10:39:00 srv-net101 librenms-service.py[1348043]: Alerting(INFO):Checking alerts
After this the Poller part stopped working but the rest of the service seems to have continued working.
I had to restart the service to get polling working again.
====================================
| Component | Version |
|---|---|
| LibreNMS | 1.70.1-29-g80ac2390c |
| DB Schema | 2020_10_12_095504_mempools_add_oids (191) |
| PHP | 7.4.3 |
| Python | 3.8.5 |
| MySQL | 10.3.25-MariaDB-0ubuntu0.20.04.1 |
| RRDTool | 1.7.2 |
| SNMP | NET-SNMP 5.8 |
| ==================================== |
[OK] Composer Version: 2.0.8
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database schema correct