mod_archive - Message Archiving (XEP-0136)

mod_archive implements XEP-0136 version 0.6 except otr (off-the-record).

Usually Jabber clients can store the conversations in the user hard drive. But sometimes this isn't possible or affordable, for example when using a mobile phone, or in a public access machine. XEP-0136 proposes that the Jabber server archives the messages as they pass through the server, if the Jabber client requested it. Later, the client can request to the server the messages. As a summary, mod_archive allows a Jabber client to request the server to store his own conversations, and later request them.

There are three different versions of this module. The main difference between them is the storage method:

  • mod_archive uses Mnesia (old)
  • mod_archive_sql uses PostgreSQL (old)
  • mod_archive_odbc uses MySQL or SQLite3, and includes other improvements
There is also the module mod_archive_webview which is a web interface to mod_archive_odbc (for vieweing the logs) You can find thoses modules in ejabberd-modules See also mod_archive_odbc homepage


Name: mod_archive
Purpose: Message archiving
Author: Olivier Goffart <ogoffart at kde.org>, Alexey Shchepin and Alexander Tsvyashchenko
Type: Module
Requirements: ejabberd 1.1.1
Download: mod_archive in ejabberd-modules

mod_archive

Features

  • Automatic archiving
  • User may enable/disable automatic archiving for one contact or globally
  • Manual archiving
  • Retreive or remove archive
  • XEP-0059

Not Supported

  • Off the record
  • Groupchats message

Options

save_default
true or false: whether or not messages should be saved by default
session_duration
The time in seconds before a session timeout (for a collection). The default value is 30 minutes.

Support of XEP-136 on Jabber clients

JWChat
Implemented, but does not work, since it implements an old version. An update on JWChat is expected in the mid-term.
Web reader
Php web interface

Related Links




Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

mod_archive crash

hi,

i want to use mod_archive module. i have downloaded ejabberd modules and follow the step in http://www.ejabberd.im/ejabberd-modules. i have generated mod_archive.beam file by compiling mod_archive.erl and put this file in Applications/ejabberd-13.12/lib/ejabberd-13.12/ebin (mac os).
but when start module i got following crash report.

2014-04-22 12:07:13 =CRASH REPORT====
crasher:
initial call: mod_archive:init/1
pid: <0.505.0>
registered_name: []
exception exit: {{{case_clause,undefined},[{gen_iq_handler,add_iq_handler,6,[{file,"src/gen_iq_handler.erl"},{line,65}]},{mod_archive,init,1,[{file,"mod_archive.erl"},{line,126}]},{gen_server,init_it,6,[{file,"gen_server.erl"},{line,304}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]},[{gen_server,init_it,6,[{file,"gen_server.erl"},{line,328}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}
ancestors: [ejabberd_sup,<0.37.0>]
messages: []
links: [<0.308.0>]
dictionary: []
trap_exit: false
status: running
heap_size: 610
stack_size: 27
reductions: 164
neighbours:

please, can anyone help me.
thanks in advance.

Problem with mod_archive_odbc and offline messages

Hi,
I set the mod_archive_odbc in my server and apparently work. But when I send a offline message to any contact, the mod_archive_odbc save in database the message, then when the contact receive the message the mod_archive_odbc save again the same message. Resulting in something like that:

+----+---------+---------------------+------+-----------------+------+
| id | coll_id | utc                 | dir  | body            | name |
+----+---------+---------------------+------+-----------------+------+
|  1 |       1 | 2012-10-25 16:35:24 |    1 | offline message |      |
|  2 |       3 | 2012-10-25 16:35:24 |    0 | offline message |      |
|  3 |       3 | 2012-10-25 16:35:32 |    0 | offline message |      |

Initially I think it's a problem in my javascript client, but I tested it in other clients and all to the same.

Now I don't know if it is a server problem or a client bad implementation problem.

Anyone have a ideia how to fix it or a better way to do that?

Thank you and sorry for my english,

Bottin

P.S.: I tryed the mod_offline and the mod_offline_odbc

Please help me to setup for store the ejabberd messages in db.

Can you please instruct me to setup the ejabberd for store the messages in my database.

I try the setup but, after that i am not able to login to the admin panel in browser or create a new user.
I follow this http://srijan.me/ejabberd-storing-data-in-mysql/ for setup the ejabberd, but not working. I also need to configure it with the MongoDB.

Your help is appreciated.

Thanks.

bottin wrote:

Hi,
I set the mod_archive_odbc in my server and apparently work. But when I send a offline message to any contact, the mod_archive_odbc save in database the message, then when the contact receive the message the mod_archive_odbc save again the same message. Resulting in something like that:

+----+---------+---------------------+------+-----------------+------+
| id | coll_id | utc                 | dir  | body            | name |
+----+---------+---------------------+------+-----------------+------+
|  1 |       1 | 2012-10-25 16:35:24 |    1 | offline message |      |
|  2 |       3 | 2012-10-25 16:35:24 |    0 | offline message |      |
|  3 |       3 | 2012-10-25 16:35:32 |    0 | offline message |      |

Initially I think it's a problem in my javascript client, but I tested it in other clients and all to the same.

Now I don't know if it is a server problem or a client bad implementation problem.

Anyone have a ideia how to fix it or a better way to do that?

Thank you and sorry for my english,

Bottin

P.S.: I tryed the mod_offline and the mod_offline_odbc

Sorry, we cannot reproduce

Sorry, we cannot reproduce your duplicate issue. This is working fine for me.

Hi, I have the same problem

Hi, I have the same problem like yours.

Have you found a way to fix it or workaround? Please share me too. Many thanks!

Hi. Seems that I have the

Hi.

Seems that I have the same problem with mod_archive
Do you know solution of this problem?

any plans to add Oracle and/or SQL Server support to mod_archive

Would like to know if there are any plans to add Oracle and/or SQL server support to mod_archive.

Also, am interested to know if DB calls are asynchronous and if there are issues w/ memory consumption under heavy load (e.g., queue of db writes grows). I know that IM compliance/proxy solutions like Actiance deal with this by writing to a persistent message queue (MSMQ) because write time to a queue is much faster than the db while having the benefit of being persistent and have a separate process to perform the db write (sometimes in bulk).

Dave

date error using postgresql as backend

i am having huge trouble getting mod_archive to work using postgresql via odbc as a backend.

when trying to retrieve messages from the archive i get the following error:

LOG:  statement: begin;
LOG:  statement: select * from archive_collections where us = 'foo' and deleted = 0 and with_user = 'bar' and with_server = 'baz' and with_resource = 'bin' and utc = '2011-04-27 20:43:12'
LOG:  statement: select count(*) from archive_messages where coll_id = 3 and utc >= '0000-01-01 00:00:00'  and utc < '2038-01-19 00:00:00' 
ERROR:  date/time field value out of range: "0000-01-01 00:00:00" at character 68
STATEMENT:  select count(*) from archive_messages where coll_id = 3 and utc >= '0000-01-01 00:00:00'  and utc < '2038-01-19 00:00:00' 
LOG:  statement: ROLLBACK
LOG:  statement: rollback;

any hint is highly appreciated.

Patch for mod_archive_webview

Patch given at http://www.ndl.kiev.ua/content/mod_archive_odbc-release#comment-265 fixes the issue because of which mod_archive_webview was not working. The patch has been tested with ejabberd 2.1.5 and mod_archive r1057 in Debian Squeeze. Here is a copy of the patch.

Index: src/mod_archive_webview.erl
===================================================================
--- src/mod_archive_webview.erl (revision 1109)
+++ src/mod_archive_webview.erl (working copy)
@@ -102,7 +102,7 @@
     make_xhtml(?T("Chat with ") ++ Jid, contact_config(Jid,US,Lang) ++
                            [?XE("ul", lists:map( fun({Id, Node, Server, Resource, Utc, Subject }) ->
                                                     With = jlib:jid_to_string({Node,Server,Resource}),
-                                                    ?LI([?AC(?LINK("show/" ++ integer_to_list(Id)), "On " ++ Utc ++ " with " ++ With ++ " -> " ++ escape_str(Subject)  )] ) end,
+                                                    ?LI([?AC(?LINK("show/" ++ Id), "On " ++ Utc ++ " with " ++ With ++ " -> " ++ escape_str(Subject)  )] ) end,
                                                 get_collection_list(jlib:string_to_jid(Jid), US)))
                ], Lang);

@@ -207,7 +207,7 @@
%------------------------

format_message({ Utc, Dir, Body } ,{WithU,WithS,WithR}, {LUser,LServer} ) ->
-    {From, Class} = case Dir of
+    {From, Class} = case list_to_integer(Dir) of
         0 -> { jlib:jid_to_string({WithU,WithS,WithR}) , "message_from" } ;
         1 -> { jlib:jid_to_string({LUser,LServer,""}) , "message_to" }
     end,
@@ -411,7 +411,7 @@

format_search_result( {Id,Subject,User,Server,Resource,Utc,Body} ,_Lang) ->
     ?XAE("p",[{"class","search_result"}],
-         [?AC(?LINK("show/" ++ integer_to_list(Id)), jlib:jid_to_string({User,Server,Resource}) ++ " : " ++ escape_str(Subject)),
+         [?AC(?LINK("show/" ++ Id), jlib:jid_to_string({User,Server,Resource}) ++ " : " ++ escape_str(Subject)),
           ?C(Body), ?XE("em",[?C(Utc)]) ] ).
          
links_previous_next({PrevId,NextId},Lang) ->
@@ -422,7 +422,7 @@
links_previous_next_aux(Class, Text, Id) ->
     case Id of
         -1 -> [];
-        _ -> [?XAE("a",[{"href",?LINK("show/" ++ integer_to_list(Id))},{"class",Class}], [?C(Text)])]
+        _ -> [?XAE("a",[{"href",?LINK("show/" ++ Id)},{"class",Class}], [?C(Text)])]
     end.

%------------------------

Internal Server Error when attempting to list a collection

Hi,
I've sometimes issues to listing a collection.
The XMPP stanzas that fail are something like this:

  <iq type='get' xmlns='jabber:client'>
   <list xmlns='http://www.xmpp.org/extensions/xep-0136.html#ns' with='test@localhost' start='2010-11-2T00:00:00.000000Z'>
     <set xmlns='http://jabber.org/protocol/rsm'>
        <max>5</max>
     </set>
    </list>
  </iq>

There are days it works like a charm and days where I get an "internal server error" as response.
After restarting the ejabberd server it works again.
I've seen a similar issue on http://www.ndl.kiev.ua/content/mod_archive_odbc-release but it was
a PosgreSQL problem.

My config is:
- ejabberd version ejabberd-2.1.3
- mod_archive_odbc at revision 1057 from https://svn.process-one.net/ejabberd-modules/mod_archive
- RDBMS MySQL 5.0.26

ejabberd.cfg

{mod_archive_odbc, [{database_type, "mysql"},
{default_auto_save, true},
{enforce_default_auto_save, true},
{default_expire, infinity},
{enforce_min_expire, 0},
{enforce_max_expire, infinity},
{replication_expire, 31536000},
{session_duration, 1800},
{wipeout_interval, 86400}]},

the config is a result of http://www.ndl.kiev.ua/content/mod_archive_odbc-release

The log looks like this :

E(<0.2977.0>:mod_archive_odbc:2428) : failed transaction: {aborted,
{{badmatch,
{aborted,
"query timed out"}},
[{mod_archive_odbc,
get_last_inserted_id,
2},
{mod_archive_odbc,
get_collection_id,
1},
{mod_archive_odbc,
new_dict_answer,
5},
{mod_archive_odbc,
'-do_log/12-fun-0-',
13},
{ejabberd_odbc,
outer_transaction,
3},
{ejabberd_odbc,
session_established,
3},
{p1_fsm,
handle_msg,10},
{proc_lib,
init_p,5}]}}, stack: {backtrace,
<<"Program counter: 0xb7c234e4 (unknown function)\nCP: 0xb30dfb00 (mod_archive_odbc:run_sql_transaction/2 + 232)\n\n0xb3effe24 Return addr 0xb30d3da4 (mod_archive_odbc:do_log/12 + 256)\ny(0) {aborted,{{badmatch,{aborted,\"query timed out\"}},[{mod_archive_odbc,get_last_inserted_id,2},{mod_archive_odbc,get_collection_id,1},{mod_archive_odbc,new_dict_answer,5},{mod_archive_odbc,'-do_log/12-fun-0-',13},{ejabberd_odbc,outer_transaction,3},{ejabberd_odbc,session_established,3},{p1_fsm,handle_msg,10},{proc_lib,init_p,5}]}}\n\n0xb3effe2c Return addr 0xb30d2a78 (mod_archive_odbc:handle_cast/2 + 460)\ny(0) []\ny(1) []\ny(2) []\ny(3) {dict,0,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}}\ny(4) []\ny(5) []\ny(6) []\ny(7) []\ny( []\ny(9) []\ny(10) []\n\n0xb3effe5c Return addr 0xb6a59d14 (gen_server:handle_msg/5 + 932)\ny(0) []\ny(1) []\ny(2) []\ny(3) []\ny(4) []\ny(5) []\ny(6) []\ny(7) []\ny( []\ny(9) []\ny(10) []\ny(11) []\ny(12) {state,\"handelsplattform.zentrale.de\",{dict,0,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},1800}\n\n0xb3effe94 Return addr 0xb6a637ec (proc_lib:init_p/5 + 200)\ny(0) mod_archive_odbc\ny(1) {state,\"handelsplattform.zentrale.de\",{dict,0,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},1800}\ny(2) 'ejabberd_mod_archive_odbc_handelsplattform.zentrale.de'\ny(3) <0.251.0>\ny(4) {'$gen_cast',{addlog,\"chat\",to,\"vmuth\",\"handelsplattform.zentrale.de\",\"40554349641283841483603324\",{jid,\"mttest\",\"handelsplattform.zentrale.de\",[],\"mttest\",\"handelsplattform.zentrale.de\",[]},[],[],[],\"wird das auf der db hinterlegt?\"}}\ny(5) Catch 0xb6a59d14 (gen_server:handle_msg/5 + 932)\n\n0xb3effeb0 Return addr 0x081dbe40 ()\ny(0) Catch 0xb6a637fc (proc_lib:init_p/5 + 216)\ny(1) gen\ny(2) init_it\ny(3) [gen_server,<0.251.0>,<0.251.0>,{local,'ejabberd_mod_archive_odbc_handelsplattform.zentrale.de'},mod_archive_odbc,[\"handelsplattform.zentrale.de\",[{database_type,\"mysql\"},{default_auto_save,true},{enforce_default_auto_save,true},{default_expire,infinity},{enforce_min_expire,0},{enforce_max_expire,infinity},{replication_expire,31536000},{session_duration,1800},{wipeout_interval,86400}]],[]]\n">>}

=ERROR REPORT==== 2010-09-07 08:39:23 ===
E(<0.2977.0>:mod_archive_odbc:489) : error when performing automated archiving: {xmlelement,
"error",
[{"code",
"500"},
{"type",
"wait"}],
[{xmlelement,
"internal-server-error",
[{"xmlns",
"urn:ietf:params:xml:ns:xmpp-stanzas"}],
[]}]}

xep-0136 capable client

For any looking for xep-0136 capable client - vacuum im (open-source, cross-platform, http://code.google.com/p/vacuum-im/) supports server-side history. I was able to build ejabberd+mod_archive_odbc+mysql and server-side history works for me now - only reasonably working client I found is vacuum im. If anyone knows more xep-0136 capable clients - please post them here.

Can you tell me how I have to

Can you tell me how I have to configure Vacuum-IM to activate that function?

You don't need to configure

You don't need to configure client for this feature to work. Just set up ejabberd+mod_archive_mysql. As long as you will see history items coming into history table in mysql - it's working.

mod_archive_webview does not work with ejabberd 2.0.1

Hi,
I am using ejabberd 2.0.1 on Debian Lenny which has been patched with mod_archive module taken from trunk. I have followed all the instructions in README.txt and http://www.ndl.kiev.ua/content/mod_archive_odbc-release for setting it up. The messages get archived properly. But when I try to access the archive via my browser using the URL http://localhost:5280/archive/ ejabberd, it throws an error as shown below in sasl.log:

=CRASH REPORT==== 13-Jul-2009::20:50:07 ===
  crasher:
    pid: <0.718.0>
    registered_name: []
    exception error: no function clause matching
                     mod_archive_webview:process([],
                                                 {request,'GET',
                                                  ["archive"],
                                                  [{nokey,[]}],
                                                  undefined,undefined,"en-us",
                                                  [],
                                                  {115,108,115,201}})
      in function  ejabberd_http:process_request/1
      in call from ejabberd_http:process_header/2
      in call from ejabberd_http:receive_headers/1
    initial call: ejabberd_http:receive_headers({state,gen_tcp,#Port<0.577>,
                                                 undefined,undefined,
                                                 undefined,undefined,
                                                 undefined,undefined,"en",
                                                 [{["archive"],
                                                   mod_archive_webview},
                                                  {["admin"],
                                                   ejabberd_web_admin},
                                                  {["http-poll"],
                                                   ejabberd_http_poll}],
                                                 false,[]})
    ancestors: [ejabberd_http_sup,ejabberd_sup,<0.37.0>]
    messages: []
    links: [<0.242.0>]
    dictionary: []
    trap_exit: false
    status: running
    heap_size: 610
    stack_size: 23
    reductions: 364
  neighbours:

=SUPERVISOR REPORT==== 13-Jul-2009::20:50:07 ===
     Supervisor: {local,ejabberd_http_sup}
     Context:    child_terminated
     Reason:     function_clause
     Offender:   [{pid,<0.718.0>},
                  {name,undefined},
                  {mfa,
                      {ejabberd_http,start_link,
                          [{gen_tcp,#Port<0.577>},
                           [http_poll,web_admin,
                            {request_handlers,
                                [{["archive"],mod_archive_webview}]}]]}},
                  {restart_type,temporary},
                  {shutdown,brutal_kill},
                  {child_type,worker}]

I tried to debug but hardly have any knowledge of erlang. Is something wrong with my configuration or is mod_archive_webview broken? Looking forward for any possible support in resolving this issue.

New version,

New version, mod_archive_odbc is available at http://www.ndl.kiev.ua/typo/articles/2007/11/14/mod_archive_odbc-release

This version, I hope, should solve quite some problems existed in earlier implementations.

errors

1) when I retrieve message page from collection I cannot retrieve next page by 'last' value (something here last value return the same page that first)

2) when I retrieve archive store, they have wrong timestamp (it is not UTC). Maybe I must to change some config settings?

I have -vsn('$Revision: 702 $ ').

[FIX] Ejabberd offline messages with wrong timestamp

To retrive archive store with correct timestamp see: http://egoncalves.com.br/ejabberd/ejabberd-offline-messages-with-wrong-t...

Storage question

Can it store the archived messages in odbc/sql/wherever, or only in mnesia? Because with current client support, (that is, lack thereof) sql storage is essential, since it's the only way to get at the logs. You can expect to turn automated logging on from an XML console, but reading them that way would be pointless.

Relationnal storage is not yet supported

Relationnal storage is not yet supported, but note that you can access Mnesia data with any Erlang program :)

(Yes, I guess that you might not want to learn Erlang for that, but data is accessible)

The main reason to use relationnal database would be probably to increase the amount of data you can put in the database.

--
Mickaël Rémond
Process-one

Is relational storage

Is relational storage support planned?

When will I can use PostgreSQL?

what is maximum amount of

what is maximum amount of data can be stored in mnesia db?

Hi, I am using ejabberd

Hi, I am using ejabberd ejabberd-2.1.5 on FreeBSD which has been patched with mod_archive module taken from trunk. I have followed all the instructions in README.txt and http://www.ndl.kiev.ua/content/mod_archive_odbc-release for setting it up.
But I have error in ejabberd.log

=ERROR REPORT==== 2011-02-17 19:46:54 ===
E(<0.36.0>:gen_mod:73) : {undef,
                             [{mod_archive_odbc,start,
                                  ["localhost",
                                   [{database_type,"mysql"},
                                    {default_auto_save,true},
                                    {enforce_default_auto_save,false},
                                    {default_expire,infinity},
                                    {enforce_min_expire,0},
                                    {enforce_max_expire,infinity},
                                    {replication_expire,31536000},
                                    {session_duration,1800},
                                    {wipeout_interval,86400}]]},
                              {gen_mod,start_module,3},
                              {lists,foreach,2},
                              {ejabberd_app,start,2},
                              {application_master,start_it_old,4}]}

Please help!...

How to access archived message at client side.

Hi,
I used text based chat functionality into my android application, and it possible by smack library for XMPP, my server also implement XEP-0136 archive protocol properly. So now i want to know the way that how to access archived message from server at client side.
please reply as quick as possible, it is very urgent.
Thanks in advance.

In accordance to XEP-0136.

In accordance to XEP-0136.

Syndicate content