Project

General

Profile

How to retrieve messages stored in database

Łukasz Soszyński
Added over 5 years ago

Hi,

I try to use message archive component in Tigase Server version 5.2.0 beta3, build 3269. Messages are correctly stored in database table tig_ma_msgs and tig_ma_jids for jid's. tig_ma_msgs content:

+----------+----------+---------------------+-----------+----------------------------------------------------+
| owner_id | buddy_id | ts                  | direction | SUBSTRING(msg, 1,50)                               |
+----------+----------+---------------------+-----------+----------------------------------------------------+
|        1 |        2 | 2013-11-08 17:15:23 |         0 | <message xmlns="jabber:client" to="psi@localhost"  |
|        1 |        2 | 2013-11-08 17:15:44 |         1 | <message xmlns="jabber:client" to="pidgin@localhos |
|        1 |        2 | 2013-11-08 17:16:13 |         1 | <message xmlns="jabber:client" to="pidgin@localhos |
|        1 |        2 | 2013-11-08 17:16:47 |         1 | <message xmlns="jabber:client" to="pidgin@localhos |
|        1 |        2 | 2013-11-08 17:17:35 |         1 | <message xmlns="jabber:client" to="pidgin@localhos |
|        1 |        2 | 2013-11-08 17:17:48 |         1 | <message xmlns="jabber:client" to="pidgin@localhos |
|        1 |        2 | 2013-11-12 12:11:58 |         0 | <message type="chat" to="psi@localhost" xmlns="jab |
|        1 |        2 | 2013-11-12 12:12:10 |         1 | <message type="chat" to="pidgin@localhost/pidginre |
|        1 |        2 | 2013-11-12 12:12:28 |         1 | <message type="chat" to="pidgin@localhost/pidginre |
+----------+----------+---------------------+-----------+----------------------------------------------------+

and tig_ma_jids content:

+--------+------------------+
| jid_id | jid              |
+--------+------------------+
|      1 | pidgin@localhost |
|      2 | psi@localhost    |
+--------+------------------+

Now I'm able to get list of chats using request:

<iq type='get' id='random_long_id_for_test_purpose_7'>
  <list xmlns='urn:xmpp:archive'
    with='psi@localhost'
    start='2012-10-01T04:00:00Z'
    end='2013-12-15T23:23:00Z'>
    <set xmlns='http://jabber.org/protocol/rsm'>
            <max>30</max>
        </set>
  </list>
</iq>

And server returns:

<iq id='random_long_id_for_test_purpose_7' type='result' to='pidgin@localhost/pidginres'>
    <list xmlns='urn:xmpp:archive'>
        <chat start='2013-11-08T00:00:00+0100' with='psi@localhost'/>
        <chat start='2013-11-12T00:00:00+0100' with='psi@localhost'/>
        <set xmlns='http://jabber.org/protocol/rsm'>
            <first index='2013-11-08T00:00:00+0100'>2013-11-08T00:00:00+0100</first>
            <last>2013-11-12T00:00:00+0100</last>
        </set>
    </list>
</iq>

Then, I want to retrieve list of messages in chat. I send request:

<iq type='get' id='random_long_id_for_test_purpose_9'>
  <retrieve xmlns='urn:xmpp:archive'
            with='psi@localhost'
            start='2013-11-12T00:00:00+0100'>
    <set xmlns='http://jabber.org/protocol/rsm'>
      <max>100</max>
    </set>
  </retrieve>
</iq>

And server returns:

<iq id='random_long_id_for_test_purpose_9' type='result' to='pidgin@localhost/pidginres'>
    <chat xmlns='urn:xmpp:archive' start='2013-11-12T00:00:00+0100' with='psi@localhost'/>
</iq>

This response is very surprising. I expected to receive list of message. What request should I send to Tigase server in order to receive list of messages in chat?


Replies (6)

Added by Andrzej Wójcik IoT 1 CloudTigaseTeam over 5 years ago

I suppose there might be an issue with retrieving archived messages if message timestamp is close to 00:00:00 in UTC and it may be related to other reported issue https://projects.tigase.org/boards/4/topics/487.

I created an issue for this and will work on solving this - https://projects.tigase.org/issues/1629

Added by Łukasz Soszyński over 5 years ago

Thank you for answering my questions.

In my database (mysql) I store messages exchanged between users in 2013-11-08 and 2013-11-12. First messages at those days were sent at 17:15:23 and 12:11:58 respectively. I wonder why Tigase provide as chat start dates: 2013-11-08T00:00:00+0100 and 2013-11-12T00:00:00+0100?

Yesterday I noticed one more issue. When I send request:

<iq type='get' id='random_long_id_for_test_purpose_7'>
  <list xmlns='urn:xmpp:archive'
    with='psi@localhost'
    start='2012-10-01T04:00:00Z'
    end='2013-12-15T23:23:00Z'>
    <set xmlns='http://jabber.org/protocol/rsm'>
              <max>30</max>
        </set>
  </list>
</iq>

all attributes (with, start, end) in list element are required. When I omit 'with' I receive response:

<error type='modify' code='406'>
    <not-acceptable xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
    <text xmlns='urn:ietf:params:xml:ns:xmpp-stanzas' lang='en'>Request parameter with must be specified</text>
</error>

When I omit dates I receive no response and NullPointerException is throw on server.

It would be very usefull for me if I can omit 'with' parameter in order to receive list of all chats with any users :)

Added by Andrzej Wójcik IoT 1 CloudTigaseTeam over 5 years ago

As you can see this is first version of Message Archiving component, and it was initialy created as an archiving mechanism for our Tigase.IM service so at first it had very limited functionality that was needed for this service to run. As we displayed there only chats with one user and those chats were grouped by day of chat Message Archiving component returns list of chats with specific user grouped by day of message - that is why you get '2013-11-12T00:00:00+1000'.

Now knowing how users try to use this component we will try to extend it and fix issues reported by users as for example add possibility to omit some field in requests to retrieve list of chats.

Added by Łukasz Soszyński about 5 years ago

Hi,

I performed similar tests on new version 5.2.0-rc1. Thank you, it works much better :) Now I can retrieve chat list (and then messages in chat) using request:

<iq type='get' id='random_long_id_for_test_purpose_7'>
  <list xmlns='urn:xmpp:archive'
    with='psi@localhost'
    start='2012-10-01T04:00:00Z'
    end='2013-12-15T23:23:00Z'>
    <set xmlns='http://jabber.org/protocol/rsm'>
              <max>30</max>
        </set>
  </list>
</iq>

It works fine. But from my point of view it is crucial to get list of chats with all users in given period. I try to omit with attribute:

<iq type='get' id='random_long_id_for_test_purpose_7'>
  <list xmlns='urn:xmpp:archive'
    start='2012-10-01T04:00:00Z'
    end='2013-12-15T23:23:00Z'>
    <set xmlns='http://jabber.org/protocol/rsm'>
              <max>30</max>
        </set>
  </list>
</iq>

but server response contains error message:

<text xml:lang="en" xmlns="urn:ietf:params:xml:ns:xmpp-stanzas">Request parameter with must be specified</text>

How can I retrieve list of chat with all users?

Added by Andrzej Wójcik IoT 1 CloudTigaseTeam about 5 years ago

I checked and database layer was ready to work without "with" attribute being passed, but there still was a condition which checked if "with" attribute was passed. I just removed this condition, so it should be ok if you use version from master branch.

Added by Łukasz Soszyński about 5 years ago

Thank You!

I will check it soon :)

    (1-6/6)