How to debug regular restarts

I have recently upgraded from 14.07 to 16.09. Following this, my client keeps disconnecting, without any reasons for this appearing in error.log or crash.log. All the logs seem normal except that the server is apparently restarting all the time. It's not due to load; there are only two users. The last line in the log will be something normal like this:

2017-01-26 12:47:13.860 [info] <0.551.0>@ejabberd_s2s_out:handle_info:940 Reconnect delay expired: Will now retry to connect to ddg.gg when needed.

And then there's about a minute of nothing and then lines showing ejabberd starting up again like this:

2017-01-26 12:47:53.525 [info] <0.7.0> Application lager started on node 'ejabberd@tardis.malcolm.id.au'

It's really hard to debug this without any errors in the logs! Where should I start?

You should investigate if the

You should investigate if the server starts for some internal reason, or influenced externally (for example some ejabberdctl or system call to restart). I mean, maybe your system has a script that checks if ejabberd works "correctly", it detects that ejabberd doesn't reply, and then restarts it.

Does it happen at certain time of day? Or when clients do something particular?

Disable the client port 5222, so your users don't connect, does it happen? Or disable port 5269 for s2s connections, does it happen? Or check if there is some cron task for checking ejabberd is running...

And if you didn't yet, set loglevel 5 so you get debug log lines

You're right, systemd is

You're right, systemd is terminating it because it thinks (wrongly) that it has failed to start:

Jan 30 12:30:15 hostname systemd[1]: Unit ejabberd.service entered failed state.
Jan 30 12:31:51 hostname systemd[1]: ejabberd.service start operation timed out. Terminating.

Thanks, looks like I just need to get systemd into line.

Syndicate content