Hello,
I recently updated ejabberd from 2.0.5 to 2.1.4 on a Gentoo server of mine.
The specs of the server are as follow :
kernel 2.6.32 (gentoo patchset) erlang 13.2.4 (with kpoll and ssl) ejabberd 2.1.4 (with pam, web, ssl, zlib) expat 2.0.1 perl 5.8.8
When I was flirting with the logs to check if the install was properly working I found a special event in my syslog:
epmd: epmd: epmd running - daemon = 1
I couldn't tell if it was already present before the update (I admit I didn't check too soon as the xmpp server was working and I don't keep more than 5 days of log on this box) but this can be quite annoying because it spams the syslog and user.log, making it grow without need.
After a bit of investigation, I found that ejabberdctl was the source of this message. Everytime I launch ejabberdctl it writes a line in the log. There it becomes a problem as my monitoring tools are extensively using ejabberdctl to get stats from the server (registered, connected, MUCs users, etc.). So I'd like to find what triggers this message.
There seems to be no crash from any component :
- epmd pid is the same before and after the message
- beam pid is the same before and after the message
- There are no message at the time of the command in ejabberd.log (when in debug log-level there are obviously messages but there aren't any warning nor error message)
So the questions are:
- Do anyone know if this message is usual, or is there an underlying error or warning behind this (quite unclear) log message?
- Is there a way to disable it or change ejabberdctl behavior so that it doesn't trigger it?
Thanks for reading my issue and thanks in advance for any answer. I will (of course) give any details needed.
arkhelion wrote: I found a
I found a special event in my syslog:
Everytime I launch ejabberdctl it writes a line in the log.
So I'd like to find what triggers this message.
There seems to be no crash from any component :
So the questions are:
1. Do anyone know if this message is usual, or is there an underlying error or warning behind this (quite unclear) log message?
2. Is there a way to disable it or change ejabberdctl behavior so that it doesn't trigger it?
This line is added by epmd when epmd is started, or when an erlang node starts and connects to epmd. Notice that ejabberd starts an erlang node, and connects to epmd.
See how the line IS added even when ejabberd isn't involved; and see that the line ISN'T added even in case of a crash in the erlang node:
$ epmd -kill epmd: Cannot connect to local epmd $ sudo tail -n 500 /var/log/syslog | grep "epmd running" | wc -l 8 $ epmd -daemon $ sudo tail -n 500 /var/log/syslog | grep "epmd running" | wc -l 9 $ epmd -names epmd: up and running on port 4369 with data: $ sudo tail -n 500 /var/log/syslog | grep "epmd running" | wc -l 9 $ epmd -help ... $ sudo tail -n 500 /var/log/syslog | grep "epmd running" | wc -l 9 $ erl -s testnode -s halt Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] {"init terminating in do_boot",{undef,[{testnode,start,[]},{init,start_it,1},{init,start_em,1}]}} Crash dump was written to: erl_crash.dump init terminating in do_boot () $ sudo tail -n 500 /var/log/syslog | grep "epmd running" | wc -l 9 $ sudo tail -n 500 /var/log/syslog | grep "epmd running" | wc -l 9 $ erl -sname abcde -s testnode -s halt Erlang R13B04 (erts-5.7.5) [source] [smp:2:2] [rq:2] [async-threads:0] [kernel-poll:false] {"init terminating in do_boot",{undef,[{testnode,start,[]},{init,start_it,1},{init,start_em,1}]}} Crash dump was written to: erl_crash.dump init terminating in do_boot () $ sudo tail -n 500 /var/log/syslog | grep "epmd running" | wc -l 10Looking at the epmd source code ( ), the log line is written by this code:
You can download the minimum erlang source code files required to compile that file, delete that line, compile epmd.c, install epmd and restart it.
Is this a wanted behavior of ejabberdctl?
This line is added by epmd when epmd is started, or when an erlang node starts and connects to epmd. Notice that ejabberd starts an erlang node, and connects to epmd.
See how the line IS added even when ejabberd isn't involved; and see that the line ISN'T added even in case of a crash in the erlang node:
<...>
Looking at the epmd source code ( ), the log line is written by this code:
You can download the minimum erlang source code files required to compile that file, delete that line, compile epmd.c, install epmd and restart it.
Thanks for your answer, so I understand that epmd is writing a line when a node connects to the daemon. The fact that I have this line when starting, restarting ejabberd is not a matter to me. But the fact that any command involving ejabberdctl triggers that line is more puzzling.
Does it mean that ejabberdctl makes the node crash? even with a simple "ejabberdctl status"? Is ejabberdctl creating his own node to get messages about the ejabberd node?
What I really mean is "is this the consequence of a wanted behavior of ejabberdctl?"
Thanks a lot for pointing me out the line in epmd.c to recompile erlang without this (debug) message, but it may also be a good thing to check if it's ok for ejabberdctl to trigger this line.
All seems correct, try the tests yourself
Is ejabberdctl creating his own node to get messages about the ejabberd node?
Right, ejabberdctl starts an erlang node that connects to the main node.
the fact that any command involving ejabberdctl triggers that line is more puzzling.
Does it mean that ejabberdctl makes the node crash? even with a simple "ejabberdctl status"?
epmd prints the debug line even when the erlang node is stopped correctly:
Even more: the line is printed when the erlang node is started, when it is still running, not when it is stopped, see:
What I really mean is "is this the consequence of a wanted behavior of ejabberdctl?"
Yes, all that I could see until now in your comments and my tests is normal operation of ejabberd.
solved!
Right, ejabberdctl starts an erlang node that connects to the main node.
This sentence answers clearly my issue, thanks a lot for pointing it to me :)
The question is then solved ^^
Thanks badlop!
Related email in the Erlang
Related threads in the Erlang mailing lists: