Home » Forums » Administrators support » Operation problem

Possible bug

Submitted by matiass32 on Wed, 2010-02-10 11:22

I'm runing ejabberd-2.1-2 on

FreeBSD it05 8.0-RELEASE FreeBSD 8.0-RELEASE #0: Sat Nov 21 15:02:08 UTC 2009 root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64

Suddenly, my /var partition is running out of space.

When I run df -h, i get:

]# df -h
Filesystem            Size    Used   Avail Capacity  Mounted on
/dev/mirror/gm0s1a    496M    373M     83M    82%    /
devfs                 1.0K    1.0K      0B   100%    /dev
/dev/mirror/gm0s1e    496M     58M    398M    13%    /tmp
/dev/mirror/gm0s1f    433G    2.6G    396G     1%    /usr
/dev/mirror/gm0s1d     13G     11G    998M    92%    /var
procfs                4.0K    4.0K      0B   100%    /proc
linprocfs             4.0K    4.0K      0B   100%    /usr/compat/linux/proc

When I cd to /var and run a du -hs I get:

# cd /var/
[root@it05 /var]# du -hs *
2.0K    account
2.0K    agentx
6.0K    at
2.0K    audit
12K    backups
4.0K    crash
6.0K    cron
168M    db
2.0K    empty
2.0K    games
2.0K    heimdal
18K    lib
8.8M    log
850K    mail
4.0K    msgs
48K    named
4.0K    net-snmp
2.0K    preserve
54K    run
2.0K    rwho
3.5M    spool
116K    tmp
24K    yp

So, it seems that something is sucking up all the space but doesn't have a file descriptor on the file system, so du is not finding the file.

Running lsof|grep /var I get:

...

beam.smp  28398 ejabberd   10w    VREG               0,93              31895    23581 /var/log/ejabberd/erlang.log
beam.smp  28398 ejabberd   11u    VREG               0,93                 97    47130 /var/spool/ejabberd/LATEST.LOG
beam.smp  28398 ejabberd   12w    VREG               0,93        11170768676    23582 /var (/dev/mirror/gm0s1d)
beam.smp  28398 ejabberd   19u    VREG               0,93             119220    47140 /var/spool/ejabberd/offline_msg.DAT
beam.smp  28398 ejabberd   20u    VREG               0,93               6809    47142 /var/spool/ejabberd/private_storage.DAT
beam.smp  28398 ejabberd   22u    VREG               0,93            1282598    47148 /var/spool/ejabberd/pubsub_item.DAT
beam.smp  28398 ejabberd   23u    VREG               0,93               5752    47154 /var/spool/ejabberd/vcard.DAT
beam.smp  28398 ejabberd   10w    VREG               0,93              31895    23581 /var/log/ejabberd/erlang.log
beam.smp  28398 ejabberd   11u    VREG               0,93                 97    47130 /var/spool/ejabberd/LATEST.LOG
beam.smp  28398 ejabberd   12w    VREG               0,93        11170768676    23582 /var (/dev/mirror/gm0s1d)
beam.smp  28398 ejabberd   19u    VREG               0,93             119220    47140 /var/spool/ejabberd/offline_msg.DAT
beam.smp  28398 ejabberd   20u    VREG               0,93               6809    47142 /var/spool/ejabberd/private_storage.DAT
beam.smp  28398 ejabberd   22u    VREG               0,93            1282598    47148 /var/spool/ejabberd/pubsub_item.DAT
beam.smp  28398 ejabberd   23u    VREG               0,93               5752    47154 /var/spool/ejabberd/vcard.DAT
beam.smp  28398 ejabberd   10w    VREG               0,93              31895    23581 /var/log/ejabberd/erlang.log
beam.smp  28398 ejabberd   11u    VREG               0,93                 97    47130 /var/spool/ejabberd/LATEST.LOG
beam.smp  28398 ejabberd   12w    VREG               0,93        11170768676    23582 /var (/dev/mirror/gm0s1d)
beam.smp  28398 ejabberd   19u    VREG               0,93             119220    47140 /var/spool/ejabberd/offline_msg.DAT
beam.smp  28398 ejabberd   20u    VREG               0,93               6809    47142 /var/spool/ejabberd/private_storage.DAT
beam.smp  28398 ejabberd   22u    VREG               0,93            1282598    47148 /var/spool/ejabberd/pubsub_item.DAT
beam.smp  28398 ejabberd   23u    VREG               0,93               5752    47154 /var/spool/ejabberd/vcard.DAT
beam.smp  28398 ejabberd   10w    VREG               0,93              31895    23581 /var/log/ejabberd/erlang.log
beam.smp  28398 ejabberd   11u    VREG               0,93                 97    47130 /var/spool/ejabberd/LATEST.LOG
beam.smp  28398 ejabberd   12w    VREG               0,93        11170768676    23582 /var (/dev/mirror/gm0s1d)
beam.smp  28398 ejabberd   19u    VREG               0,93             119220    47140 /var/spool/ejabberd/offline_msg.DAT
beam.smp  28398 ejabberd   20u    VREG               0,93               6809    47142 /var/spool/ejabberd/private_storage.DAT
beam.smp  28398 ejabberd   22u    VREG               0,93            1282598    47148 /var/spool/ejabberd/pubsub_item.DAT
beam.smp  28398 ejabberd   23u    VREG               0,93               5752    47154 /var/spool/ejabberd/vcard.DAT
beam.smp  28398 ejabberd   10w    VREG               0,93              31895    23581 /var/log/ejabberd/erlang.log
beam.smp  28398 ejabberd   11u    VREG               0,93                 97    47130 /var/spool/ejabberd/LATEST.LOG
beam.smp  28398 ejabberd   12w    VREG               0,93        11170768676    23582 /var (/dev/mirror/gm0s1d)
beam.smp  28398 ejabberd   19u    VREG               0,93             119220    47140 /var/spool/ejabberd/offline_msg.DAT
beam.smp  28398 ejabberd   20u    VREG               0,93               6809    47142 /var/spool/ejabberd/private_storage.DAT
beam.smp  28398 ejabberd   22u    VREG               0,93            1282598    47148 /var/spool/ejabberd/pubsub_item.DAT
beam.smp  28398 ejabberd   23u    VREG               0,93               5752    47154 /var/spool/ejabberd/vcard.DAT
beam.smp  28398 ejabberd   10w    VREG               0,93              31895    23581 /var/log/ejabberd/erlang.log
beam.smp  28398 ejabberd   11u    VREG               0,93                 97    47130 /var/spool/ejabberd/LATEST.LOG
beam.smp  28398 ejabberd   12w    VREG               0,93        11170768676    23582 /var (/dev/mirror/gm0s1d)
beam.smp  28398 ejabberd   19u    VREG               0,93             119220    47140 /var/spool/ejabberd/offline_msg.DAT
beam.smp  28398 ejabberd   20u    VREG               0,93               6809    47142 /var/spool/ejabberd/private_storage.DAT
beam.smp  28398 ejabberd   22u    VREG               0,93            1282598    47148 /var/spool/ejabberd/pubsub_item.DAT
beam.smp  28398 ejabberd   23u    VREG               0,93               5752    47154 /var/spool/ejabberd/vcard.DAT
beam.smp  28398 ejabberd   10w    VREG               0,93              31895    23581 /var/log/ejabberd/erlang.log
beam.smp  28398 ejabberd   11u    VREG               0,93                 97    47130 /var/spool/ejabberd/LATEST.LOG
beam.smp  28398 ejabberd   12w    VREG               0,93        11170768676    23582 /var (/dev/mirror/gm0s1d)
beam.smp  28398 ejabberd   19u    VREG               0,93             119220    47140 /var/spool/ejabberd/offline_msg.DAT
beam.smp  28398 ejabberd   20u    VREG               0,93               6809    47142 /var/spool/ejabberd/private_storage.DAT
beam.smp  28398 ejabberd   22u    VREG               0,93            1282598    47148 /var/spool/ejabberd/pubsub_item.DAT
beam.smp  28398 ejabberd   23u    VREG               0,93               5752    47154 /var/spool/ejabberd/vcard.DAT
beam.smp  28398 ejabberd   10w    VREG               0,93              31895    23581 /var/log/ejabberd/erlang.log
beam.smp  28398 ejabberd   11u    VREG               0,93                 97    47130 /var/spool/ejabberd/LATEST.LOG
beam.smp  28398 ejabberd   12w    VREG               0,93        11170768676    23582 /var (/dev/mirror/gm0s1d)
beam.smp  28398 ejabberd   19u    VREG               0,93             119220    47140 /var/spool/ejabberd/offline_msg.DAT
beam.smp  28398 ejabberd   20u    VREG               0,93               6809    47142 /var/spool/ejabberd/private_storage.DAT
beam.smp  28398 ejabberd   22u    VREG               0,93            1282598    47148 /var/spool/ejabberd/pubsub_item.DAT
beam.smp  28398 ejabberd   23u    VREG               0,93               5752    47154 /var/spool/ejabberd/vcard.DAT
beam.smp  28398 ejabberd   10w    VREG               0,93              31895    23581 /var/log/ejabberd/erlang.log
beam.smp  28398 ejabberd   11u    VREG               0,93                 97    47130 /var/spool/ejabberd/LATEST.LOG
beam.smp  28398 ejabberd   12w    VREG               0,93        11170768676    23582 /var (/dev/mirror/gm0s1d)
beam.smp  28398 ejabberd   19u    VREG               0,93             119220    47140 /var/spool/ejabberd/offline_msg.DAT
beam.smp  28398 ejabberd   20u    VREG               0,93               6809    47142 /var/spool/ejabberd/private_storage.DAT
beam.smp  28398 ejabberd   22u    VREG               0,93            1282598    47148 /var/spool/ejabberd/pubsub_item.DAT
beam.smp  28398 ejabberd   23u    VREG               0,93               5752    47154 /var/spool/ejabberd/vcard.DAT
beam.smp  28398 ejabberd   10w    VREG               0,93              31895    23581 /var/log/ejabberd/erlang.log
beam.smp  28398 ejabberd   11u    VREG               0,93                 97    47130 /var/spool/ejabberd/LATEST.LOG
beam.smp  28398 ejabberd   12w    VREG               0,93        11170768676    23582 /var (/dev/mirror/gm0s1d)
beam.smp  28398 ejabberd   19u    VREG               0,93             119220    47140 /var/spool/ejabberd/offline_msg.DAT
beam.smp  28398 ejabberd   20u    VREG               0,93               6809    47142 /var/spool/ejabberd/private_storage.DAT
beam.smp  28398 ejabberd   22u    VREG               0,93            1282598    47148 /var/spool/ejabberd/pubsub_item.DAT
beam.smp  28398 ejabberd   23u    VREG               0,93               5752    47154 /var/spool/ejabberd/vcard.DAT
beam.smp  28398 ejabberd   10w    VREG               0,93              31895    23581 /var/log/ejabberd/erlang.log
beam.smp  28398 ejabberd   11u    VREG               0,93                 97    47130 /var/spool/ejabberd/LATEST.LOG
beam.smp  28398 ejabberd   12w    VREG               0,93        11170768676    23582 /var (/dev/mirror/gm0s1d)
beam.smp  28398 ejabberd   19u    VREG               0,93             119220    47140 /var/spool/ejabberd/offline_msg.DAT
beam.smp  28398 ejabberd   20u    VREG               0,93               6809    47142 /var/spool/ejabberd/private_storage.DAT
beam.smp  28398 ejabberd   22u    VREG               0,93            1282598    47148 /var/spool/ejabberd/pubsub_item.DAT
beam.smp  28398 ejabberd   23u    VREG               0,93               5752    47154 /var/spool/ejabberd/vcard.DAT
beam.smp  28398 ejabberd   10w    VREG               0,93              31895    23581 /var/log/ejabberd/erlang.log
beam.smp  28398 ejabberd   11u    VREG               0,93                 97    47130 /var/spool/ejabberd/LATEST.LOG
beam.smp  28398 ejabberd   12w    VREG               0,93        11170768676    23582 /var (/dev/mirror/gm0s1d)
beam.smp  28398 ejabberd   19u    VREG               0,93             119220    47140 /var/spool/ejabberd/offline_msg.DAT
beam.smp  28398 ejabberd   20u    VREG               0,93               6809    47142 /var/spool/ejabberd/private_storage.DAT
beam.smp  28398 ejabberd   22u    VREG               0,93            1282598    47148 /var/spool/ejabberd/pubsub_item.DAT
beam.smp  28398 ejabberd   23u    VREG               0,93               5752    47154 /var/spool/ejabberd/vcard.DAT
beam.smp  28398 ejabberd   10w    VREG               0,93              31895    23581 /var/log/ejabberd/erlang.log
beam.smp  28398 ejabberd   11u    VREG               0,93                 97    47130 /var/spool/ejabberd/LATEST.LOG
beam.smp  28398 ejabberd   12w    VREG               0,93        11170768676    23582 /var (/dev/mirror/gm0s1d)
beam.smp  28398 ejabberd   19u    VREG               0,93             119220    47140 /var/spool/ejabberd/offline_msg.DAT
beam.smp  28398 ejabberd   20u    VREG               0,93               6809    47142 /var/spool/ejabberd/private_storage.DAT
beam.smp  28398 ejabberd   22u    VREG               0,93            1282598    47148 /var/spool/ejabberd/pubsub_item.DAT
beam.smp  28398 ejabberd   23u    VREG               0,93               5752    47154 /var/spool/ejabberd/vcard.DAT

...

So, it seems that ejabberd is sucking up 10Gb I'm missing on my filesystem, and that space is being reserved into the /var directory.

Is this a bug on ejabberd?

Is there any way to solve this without having to restart the server (not tried yet, as many people is using it....)

Thanks.

‹ LDAP auth - not displayed registered users in web admin How to remove ca. 30K spam accounts? ›
»

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

This person cross-posted the

Submitted by badlop on Sat, 2010-02-13 09:36.

This person cross-posted the same question, at the same time, to the ejabberd mailing list. But forgot to mention this fact in both submissions.

See: High filesystem usage

In that discussion, other people noticed that the problem was caused by a mistake in the administrator itself, not a bug in ejabberd.

»
Syndicate content