mod_statsdx - Advanced Statistics

This group of modules provide features related to statistics.

For ejabberd 2.0.x, use the code in mod_statsdx/trunk. For ejabberd 1.1.3 or older, use the code in mod_statsdx/branches/1.1.3


Name: mod_statsdx
Purpose: Calculate and gather a great number of statistics
Author: Badlop
Type: Module
Requirements:
Download: ejabberd-modules
This module also provides several new statistics in the ejabberd Web Admin:
Name: mod_stats2file
Purpose: Generates files periodically with all kind of statistics
Author: Badlop
Type: Module
Requirements: mod_statsdx, recommended: web server or RRDTool script
Download: ejabberd-modules

This module writes a file with all kind of statistics every few minutes. Available output formats are html (example), text file with descriptions and raw text file (for MRTG, RRDTool...).

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

mod_statsdx with traffic stats on multiple node cluster

Hi,

I'm trying to add mod_statsdx to a multi-node cluster of ejabberd 2.0.5 machines. I wanted to add the traffic stats so I did:

{mod_statsdx, [{hooks, traffic}]}

But that just made the module not load. I had to set it to {mod_statsdx, [{hooks, true}]} (but then, there are no traffic stats.) How do I enable the statistics for traffic?

Also, do I have to run this on all my nodes or just one of them?

Thanks!

Tdm4

traffic not available for 2.0.x

If you have ejabberd 2.0.5, then you must install mod_statsdx/branches/ejabberd-2.0.x. And that version does not support the option {hooks, traffic}.

More bad news for you: I've noticed that mod_statsdx/trunk does not work correctly with ejabberd 2.0.5.

Hi.

I got branches compiled and installed. I've been playing with using mod_stats2file with dat type. I'm just wondering how do I pipe that into rrdtool to generate mrtg graphs? I just want to keep track of the online user count. Is there an easy way to set rrdtool up to read that value and record it when it changes? I could use a cron job wrapper script but it may not be synced with the time the stats file gets updated. Maybe I don't need the .dat file? I could just query ejabberctl directly?

Thanks!

tdm4

ejabberdctl connected_users_number

In your case it's easier to just call:

$ ejabberdctl connected_users_number
230

That works like a charm!

That works like a charm! Thanks!
There's no way to keep track of the number of messages users are sending? (Not iq, or presence stanzas, but actual sending of messages?)
Would be nice to chart that, but I guess it doesn't work with 2.0.xx yet?

"way to keep track of the

"way to keep track of the number of messages users are sending"

That are outgoing message stanzas. In mod_statsdx trunk it can be enabled with the option {hooks, traffic}. But I didn't try that in a real server, so I'm not sure how much CPU and RAM overhead it introduces. Also, it is not available in branch 2.0.x.

mod_stats2file does not alter "Registered users"

Hello all,

I'm using ejabberd 2.0.3 Debian and mod_statsdx/mod_stats2file.

In the files mod_stats2file is creating I always see the same number of "Registered users" over the last days - since I implemented the module.

Although the number of registered users is rising permanently.

Regards,
Martin

Registered users of server only checks internal database

The number of registered accounts provided to mod_stats2file by mod_statsdx is calculated:

  • For a vhost (stats-vhost-localhost.html): the corresponding authentication system is used.
  • For the whole server (stats-server-.html): only the internal database is checked. So, if some vhost uses a different auth method, such users are not counted in this statistic.

So, the value of "Registered users" in the global server is only accurate when all vhosts use internal auth.

Re: Registered users of server only checks internal database

I see, thank you for the explanation!

I checked the stats-vhost-jabberdomain.tld.html files and found accurate numbers of users for our two secondary domains. The primary domain with almost 3000 users is not shown correct, I only see 0 registered users. All three domains use MySQL as database and I cannot think of anything which is different in the configuration which can affect this.

My error or is there a bug?

Regards,
Martin

any plans about update

any plans about update mod_statsdx for ejabberd 2.*?
at least stats to file...

Modules work with ejabberd 2.0.x

I briefly tested mod_statsdx and mod_stats2file with ejabberd trunk (similar to ejabberd 2.0.2) and they seemed to work correctly. I've also updated the documentation, check SVN.

thanks, it works! previously

thanks, it works! previously i'm cp mod_stats2file.erl & mod_statsdx.erl to /src - ejabberd don't compile, after i'm cp mod_statsdx.erl to /src/web all works fine.

README says:

hooks: Set to 'true' to enable hooks and related statistics.
This option by default 'false' because it is expected to
consume many resources in very populated servers.

how many in server with ~500-1000 active users?

mod_statsdx updated, tested with hooks: success

evadim wrote:

how many in server with ~500-1000 active users?

I've updated mod_statsdx in SVN. It has been running with {hooks, true} for several days in a production ejabberd server (500 concurrent users aprox). So hooks work correctly for me now :)

I didn't measure the resource consumption of hooks. If you try the module with hooks, and you notice beam consumes much more CPU or RAM, please tell me.

Don't enable mod_statsdx hooks yet.

evadim wrote:

README says:

hooks: Set to 'true' to enable hooks and related statistics.
This option by default 'false' because it is expected to
consume many resources in very populated servers.

how many in server with ~500-1000 active users?

Better do not enable the hooks option at all. I tried with just 300 concurrent users and after some minutes the module stopped. I'll try again the next week, and hopefully I'll be able to commit an improvement. You'll notice in the SVN log when the option is safe to be used.

thanks

thanks

ejabberd 1.1.4 + mod_statsdx

Do you plan to update mod_statsdx for ejabberd 1.1.4?
Thanks/Спасибо заранее :)

Should work

Makc666 wrote:

Do you plan to update mod_statsdx for ejabberd 1.1.4?

mod_statsdx is supposed to work correctly with ejabberd 1.1.3. ejabberd 1.1.3 and 1.1.4 are very similar. So mod_statsdx should work correctly with ejabberd 1.1.4 too.

Did you try and it didn't work correctly? Any error message in the log files?

statsdx

problem fixed... nothing to see here :)

error in mod_statsdx,

i added "{mod_statsdx, [{hooks, false}]}," and "{mod_stats2file, [{interval, 4}, {type, txt}, {basefilename, "/root/jabber"}]}," (without the double quotes) to ejabberd.cfg. now when i restart the server i get this error message

--------
Error in process <0.649.0> on node 'ejabberd@linux' with exit value: {badarg, [{ets,update_counter, [stats,{user_login,server},{2,1,0,0}]}, {mod_statsdx,get_stat,2}, {mod_stats2file,fwttl,4}, {mod_stats2file,write_stats,5}, {mod_stats2file,write_statsfiles,5}, {mod_stats2file,loop,5}]}
--------

i guess mod_stats2file is working cox /root/jabber.txt file is created with the stats. but there is nothing added to /admin/server/linux.temp/stats/ interface

i checked out /admin/server/linux.temp/node/ejabberd@linux/modules/ entries for both the modules is there. when i hit restart for "mod_statsdx" i get under mentioned error in the log file and entry for mod_statsdx disappears from the list.

--------
=ERROR REPORT==== 2007-03-27 01:27:41 ===
E(<0.356.0>:gen_mod:47) : {badarg,[{ets,
insert,
[stats,
{{user_login,"linux.temp"},"disabled"}]},
{mod_statsdx,start,2},
{gen_mod,start_module,3},
{rpc,local_call,3},
{ejabberd_web_admin,
'-node_modules_parse_query/4-fun-0-',
4},
{lists,foreach,2},
{ejabberd_web_admin,
node_modules_parse_query,
4},
{ejabberd_web_admin,get_node,5}]}
--------

ejd 1.1.2, redhat9Install

ejd 1.1.2, redhat9

Install you addon... result...

add "{mod_statsdx, []}," in ejabberd.cfg/modules

/admin/stats/
work beautifully

work only first time...
/admin/node/mydomen/stats/

not work
/admin/server/mydomen/stats/

add "{mod_statsdx, [{hooks,false}]}," in ejabberd.cfg/modules
restart server
ALL stats in not work

Check log files

What does the browser show you when they don't work?

All of them work for me, with and without hooks. Check the ejabberd log files for ERROR or INFO REPORT, they will help to find the problem.

sasl.log =CRASH REPORT====

sasl.log

=CRASH REPORT==== 19-Dec-2006::15:49:34 ===
crasher:
pid: <0.480.0>
registered_name: []
error_info: {badarg,[{ets,safe_fixtable,[irc_connection,true]},
{ets,foldl,3},
{mod_statsdx,get_irccons,1},
{ejabberd_web_admin,get_stat_v,2},
{ejabberd_web_admin,do_stat,4},
{ejabberd_web_admin,get_statsdx,2},
{ejabberd_web_admin,process_admin,2},
{ejabberd_http,process_request,1}]}
initial_call: {ejabberd_http,receive_headers,
[{state,
tls,
{tlssock,#Port<0.508>,#Port<0.509>},
undefined,
undefined,
undefined,
undefined,
undefined,
undefined,
"en",
false,
false,
true,
false,
[]}]}
ancestors: [ejabberd_http_sup,ejabberd_sup,<0.37.0>]
messages: []
links: [<0.66.0>,<0.260.0>,#Port<0.508>]
dictionary: []
trap_exit: false
status: running
heap_size: 4181
stack_size: 21
reductions: 123447
neighbours:

=SUPERVISOR REPORT==== 19-Dec-2006::15:49:34 ===
Supervisor: {local,ejabberd_http_sup}
Context: child_terminated
Reason: {badarg,[{ets,safe_fixtable,[irc_connection,true]},
{ets,foldl,3},
{mod_statsdx,get_irccons,1},
{ejabberd_web_admin,get_stat_v,2},
{ejabberd_web_admin,do_stat,4},
{ejabberd_web_admin,get_statsdx,2},
{ejabberd_web_admin,process_admin,2},
{ejabberd_http,process_request,1}]}
Offender: [{pid,<0.480.0>},
{name,undefined},
{mfa,{ejabberd_http,start_link,
[{gen_tcp,#Port<0.508>},
[web_admin,
tls,
{certfile,
"/var/lib/ejabberd/ssl/server.pem"}]]}},
{restart_type,temporary},
{shutdown,brutal_kill},
{child_type,worker}]

server not configure irc...

{modules,
[
...
% {mod_irc, []},
....
]}.

i see {mod_statsdx,get_irccons,1},
and test you patch get(_, ["ircconns", Host]) -> 0; in line 334!!
its not work addition
/admin/stats/

if add {mod_irc, []} when mod_statsdx work :(

hm.... all over again...

hm.... all over again... work only first time... afterwards

/admin/stats/
/admin/server/benladen.uezku.net/stats/

similar error in sasl.log

=CRASH REPORT==== 20-Dec-2006::16:21:51 ===
crasher:
pid: <0.4743.0>
registered_name: []
error_info: {badarg,[{ets,lookup,
[stats,{client,"benladen.uezku.net",bitlbee}]},
{mod_statsdx,get,2},
{mod_statsdx,'-get/2-fun-0-',3},
{lists,map,2},
{ejabberd_web_admin,do_stat_table,4},
{ejabberd_web_admin,get_statsdx,2},
{ejabberd_web_admin,process_admin,2},
{ejabberd_http,process_request,1}]}
initial_call: {ejabberd_http,receive_headers,
[{state,
tls,
{tlssock,#Port<0.3681>,#Port<0.3682>},
undefined,
undefined,
undefined,
undefined,
undefined,
undefined,
"en",
false,
false,
true,
false,
[]}]}
ancestors: [ejabberd_http_sup,ejabberd_sup,<0.37.0>]
messages: []
links: [<0.66.0>,<0.272.0>,#Port<0.3681>]
dictionary: []
trap_exit: false
status: running
heap_size: 2584
stack_size: 21
reductions: 83519
neighbours:

=SUPERVISOR REPORT==== 20-Dec-2006::16:21:51 ===
Supervisor: {local,ejabberd_http_sup}
Context: child_terminated
Reason: {badarg,[{ets,lookup,
[stats,{client,"benladen.uezku.net",bitlbee}]},
{mod_statsdx,get,2},
{mod_statsdx,'-get/2-fun-0-',3},
{lists,map,2},
{ejabberd_web_admin,do_stat_table,4},
{ejabberd_web_admin,get_statsdx,2},
{ejabberd_web_admin,process_admin,2},
{ejabberd_http,process_request,1}]}
Offender: [{pid,<0.4743.0>},
{name,undefined},
{mfa,{ejabberd_http,start_link,
[{gen_tcp,#Port<0.3681>},
[web_admin,
tls,
{certfile,
"/var/lib/ejabberd/ssl/server.pem"}]]}},
{restart_type,temporary},
{shutdown,brutal_kill},
{child_type,worker}]

virtual host stats not work

I have the same problem , virtual host not work.
I can view this in sasl.log

=CRASH REPORT==== 12-Dec-2006::12:42:05 ===
crasher:
pid: <0.1283.0>
registered_name: []
error_info: {badarg,[{ets,safe_fixtable,[irc_connection,true]},
{ets,foldl,3},
{mod_statsdx,get_irccons,1},
{ejabberd_web_admin,get_stat_v,2},
{ejabberd_web_admin,do_stat,4},
{ejabberd_web_admin,get_statsdx,2},
{ejabberd_web_admin,process_admin,2},
{ejabberd_http,process_request,1}]}
initial_call: {ejabberd_http,receive_headers,
[{state,
gen_tcp,
#Port<0.838>,
undefined,
undefined,
undefined,
undefined,
undefined,
undefined,
"en",
true,
true,
true,
false,
[]}]}
ancestors: [ejabberd_http_sup,ejabberd_sup,<0.805.0>]
messages: []
links: [<0.834.0>,<0.1003.0>,#Port<0.838>]
dictionary: []
trap_exit: false
status: running
heap_size: 1597
stack_size: 21
reductions: 3926
neighbours:

=SUPERVISOR REPORT==== 12-Dec-2006::12:42:05 ===
Supervisor: {local,ejabberd_http_sup}
Context: child_terminated
Reason: {badarg,[{ets,safe_fixtable,[irc_connection,true]},
{ets,foldl,3},
{mod_statsdx,get_irccons,1},
{ejabberd_web_admin,get_stat_v,2},
{ejabberd_web_admin,do_stat,4},
{ejabberd_web_admin,get_statsdx,2},
{ejabberd_web_admin,process_admin,2},
{ejabberd_http,process_request,1}]}
Offender: [{pid,<0.1283.0>},
{name,undefined},
{mfa,{ejabberd_http,start_link,
[{gen_tcp,#Port<0.838>},
[http_poll,http_bind,web_admin]]}},
{restart_type,temporary},
{shutdown,brutal_kill},
{child_type,worker}]

=CRASH REPORT==== 12-Dec-2006::12:42:05 ===
crasher:
pid: <0.1284.0>
registered_name: []
error_info: {badarg,[{ets,safe_fixtable,[irc_connection,true]},
{ets,foldl,3},
{mod_statsdx,get_irccons,1},
{ejabberd_web_admin,get_stat_v,2},
{ejabberd_web_admin,do_stat,4},
{ejabberd_web_admin,get_statsdx,2},
{ejabberd_web_admin,process_admin,2},
{ejabberd_http,process_request,1}]}
initial_call: {ejabberd_http,receive_headers,
[{state,
gen_tcp,
#Port<0.839>,
undefined,
undefined,
undefined,
undefined,
undefined,
undefined,
"en",
true,
true,
true,
false,
[]}]}
ancestors: [ejabberd_http_sup,ejabberd_sup,<0.805.0>]
messages: []
links: [<0.834.0>,<0.1003.0>,#Port<0.839>]
dictionary: []
trap_exit: false
status: running
heap_size: 1597
stack_size: 21
reductions: 3926
neighbours:

=SUPERVISOR REPORT==== 12-Dec-2006::12:42:05 ===
Supervisor: {local,ejabberd_http_sup}
Context: child_terminated
Reason: {badarg,[{ets,safe_fixtable,[irc_connection,true]},
{ets,foldl,3},
{mod_statsdx,get_irccons,1},
{ejabberd_web_admin,get_stat_v,2},
{ejabberd_web_admin,do_stat,4},
{ejabberd_web_admin,get_statsdx,2},
{ejabberd_web_admin,process_admin,2},
{ejabberd_http,process_request,1}]}
Offender: [{pid,<0.1284.0>},
{name,undefined},
{mfa,{ejabberd_http,start_link,
[{gen_tcp,#Port<0.839>},
[http_poll,http_bind,web_admin]]}},
{restart_type,temporary},
{shutdown,brutal_kill},
{child_type,worker}]

quick try

ppcharli wrote:

error_info: {badarg,[{ets,safe_fixtable,[irc_connection,true]},

Maybe the problem is calculating the IRC stats. Try to modify the line 318 on mod_statsdx.erl and put:

get(_, ["ircconns", Host]) -> 0;

work perfect

OK with this modification it works perfect.
Thanks

mod_stats2file with 1.1.2 with mod_roster_odbc

I'm using mod_stats2file with ejabberd 1.1.2 with mod_roster_odbc (mysql).

It appears as though when mod_stats2file hits the roster output, it errors. Commenting out mod_stats2file seems to make the error go away.

The error is:

ejabberd.log:Error in process <0.3323.0> on node 'ejabberd@systems' with exit value: {{badmatch,{aborted,{no_exists,roster}}}, [{mod_statsdx,get_totalrosteritems,1}, {mod_stats2file,fwttl,4}, {mod_stats2file,write_stats,5}, {lists,foreach,2}, {mod_stats2file,write_statsfiles,5}, {mod_stats2file,loop,5}]}

I'm guessing by the {no_exists,roster} that this probably has to do with me using mysql roster.

Also, the stats output file ends like this:

<h2>Accounts</h2>
<blockquote>
Registered users: 311<br/>
</blockquote>
<h2>Roster</h2>
<blockquote>

And that's the end of the file.

Any thoughts? Thanks.

Right, only internal Mnesia database is used

jasonaward wrote:

I'm using mod_stats2file with ejabberd 1.1.2 with mod_roster_odbc (mysql).
...
I'm guessing by the {no_exists,roster} that this probably has to do with me using mysql roster.

You are right, mod_statsdx gets stats from ejabberd, Mnesia and ETS. If you use another database (ODBC, MySQL...) some stats can't be calculated. It would be nice if this feature were implemented, but I don't have knowledge, or any special interest in doing it right now.

The quickest solution to the error is to comment the code. If you have a patch that implements that feature, tell me :)

With ejabberd 1.1.2

Hi,

Is it still working with ejabberd 1.1.2 ?

I've just tryed to use it with this release and i still get the old simple stats interface (and yes I do things right in conf)

Thanks

--
beber@meleeweb.net

Confirmed to work on 1.1.2

Yes, they work with ejabberd 1.1.2. Make sure you follow the steps:

  1. Apply the patch: patch -p0<ejabberd.....diff
  2. Copy the module to src dir
  3. Compile ejabberd
  4. Configure the module on ejabberd.cfg

Things to check:

  • Check log files, there should not be any ERROR or CRASH reports
  • Check on the web interface -> virtual host -> node -> modules, that mod_statsdx is really enabled.

I have: FreeBSD 6.0 ejabberd

I have:
FreeBSD 6.0
ejabberd 1.1.1+mod_statsdx+mod_stats2file

in ejabberd.cfg

{mod_stats2file, [{interval, 4}, {type, txt}, {basefilename, "/usr/local/www/data-dist/inc/jabber"}]},

The file jabber.txt has been created at start Ejabberd, but it is not updated more

any ideas?

ODBC

Hello
I've installed mod_statsdx to my ejabberd(2.1.11) but I've discovered that it cannot give statistics about the items which are stored in mysql.
I'm using ejabberd_odbc libraries and storing user messages in mysql so mod_statsdx is not able to retrieve information about the messages users send or received.
Do you know is there any solution to that or will there be any bugfix for that?
Or if you want to use mod_statsdx do you have to use mnesia database?

Thanks
Tolga

Syndicate content