Project

General

Profile

Message archiving and XEP-0004: Data Forms. How to retrieve custom fields?

Łukasz Soszyński
Added over 4 years ago

Hi,

I try to archive custom messages using Message Archiving component. At the beginning I try to archive messages in format:

<message to='20@localhost' from='1@localhost' id='dyP9g-18'>
    <subject>Hi</subject>
    <body>Message body</body>
    <custom xmlns='http://my.namespace.com'>
        <name>000000003659</name>
         <surname>15</surname>
        <date>1374144203775</date>
    </custom>
</message>

The above message is correctly stored in database and delivered to recipient. But when I retrieve messages via archiving component then custom fields disappear. I thought that it may be caused by custom nature of my message so I decide to use XEP-0004 standard (Data Forms). But behaviour is the same. Here is a detailed description:

  • My database content (sorry for formatting):

+----------+----------+---------------------+-----------+------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

| owner_id | buddy_id | ts | direction | type | msg |
+----------+----------+---------------------+-----------+------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

| 3 | 4 | 2014-03-11 15:34:17 | 0 | chat |
<message type="chat" time="1394548457051" xmlns="jabber:client"

to="20@localhost/dev-VirtualBox" from="1@localhost/pidgin"

id="purplee7f6506f">

Hello my friend. Please fill out the form :)

|

| 3 | 4 | 2014-03-11 15:38:19 | 0 | NULL |

<subject>Messagw with XEP-0004</subject>

<body>Contains Data Forms</body>

    <x type="submit" xmlns="jabber:x:data">

             <field type="text-single" var="name">

                    <value>James</value>

          </field>

            <field type="text-single" var="surname">

                    <value>Bond</value>

          </field>

            <field type="text-single" var="date">

                    <value>1374144203775</value>

            </field>

    </x>

|

+----------+----------+---------------------+-----------+------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

  • My XMPP request for getting archived messages:
<!-- Get list of chats, request-->
<iq type='get' id='id0001'>
  <list xmlns='urn:xmpp:archive'
    start='2010-10-01T04:00:00Z'
    end='2015-12-15T23:23:00Z'>
    <set xmlns='http://jabber.org/protocol/rsm'>
              <max>30</max>
        </set>
  </list>
</iq>

<!-- Chats list, server response-->
<iq type='result' to='1@localhost/pidgin' id='id0001'>
    <list xmlns='urn:xmpp:archive'>
        <chat with='20@localhost' start='2014-03-11T14:34:17+0000'/>
        <set xmlns='http://jabber.org/protocol/rsm'>
            <first index='0'>2014-03-11T14:34:17+0000</first>
            <last>2014-03-11T14:34:17+0000</last>
            <count>1</count>
        </set>
    </list>
</iq>

<!-- Get messages from chat request-->
<iq type='get' id='id0002'>
  <retrieve xmlns='urn:xmpp:archive'
            with='20@localhost'
            start='2014-03-11T14:34:17+0000'>
    <set xmlns='http://jabber.org/protocol/rsm'>
      <max>100</max>
    </set>
  </retrieve>
</iq>

<!-- Messages within chat, server response-->
<iq type='result' to='1@localhost/pidgin' id='id0002'>
    <chat xmlns='urn:xmpp:archive' with='20@localhost' start='2014-03-11T14:34:17+0000'>
        <to secs='0'>
            <body>Hello my friend. Please fill out the form :)</body>
        </to>
                <!-- Message without form fields-->
        <to secs='242'>
            <body>Contains Data Forms</body>
        </to>
        <set xmlns='http://jabber.org/protocol/rsm'>
            <first index='0'>0</first>
            <last>242</last>
            <count>2</count>
        </set>
    </chat>
</iq>

Now you can see that second message contains no XEP-0004 form fields. Is it possible to retrieve via archiving component original message? I can use first or second approach.

P.S.

I am using Tigase 5.2.0 version and in reality data inside custom messages are a little bit more complicated.


Replies (9)

Added by Andrzej Wójcik IoT 1 CloudTigaseTeam over 4 years ago

Currently Message Archiving component implementing XEP-0136 stores full message stanza in database but it is only possible to retrieve <body/> part of a message. It is possible to extend it be able to retrieve whole message stanzas as chats items, but it is not implemented right now, as Message Archiving component was considered to be a component which stores only chat messages, and due to that it requires message to contain <body/> element.

If you would like to see this feature added in next release, fill free and create a new issue for Tigase Message Archiving project.

Avatar?id=6023&size=32x32

Added by Artur Hefczyc TigaseTeam over 4 years ago

Thank you, I assigned Andrzej to the ticket as he is the component author.

Added by Łukasz Soszyński over 4 years ago

According to comment in issue tracker [[[https://projects.tigase.org/issues/1777]]] if full message is stored in database then full message will be returned to client.

SENT:

<message type="chat" to="000000000101@localhost/localhost" id="ab2easfsfd">
<body>aaaaaabbbbbbbbcccccccc</body>
<x type="submit" xmlns="jabber:x:data">
                 <field type="text-single" var="name">
                        <value>James</value>
              </field>
                <field type="text-single" var="surname">
                        <value>Bond</value>
              </field>
                <field type="text-single" var="date">
                        <value>1374144203775</value>
                </field>
        </x>
<active xmlns="http://jabber.org/protocol/chatstates"/>
</message>

IN DB:

<message id="ab2easfsfd" type="chat" xmlns="jabber:client" from="000000000131@localhost/dev" time="1396275015536" to="000000000101@localhost/localhost">
<body>aaaaaabbbbbbbbcccccccc</body>
<x type="submit" xmlns="jabber:x:data">
                 <field var="name" type="text-single">
                        <value>James</value>
              </field>
                <field var="surname" type="text-single">
                        <value>Bond</value>
              </field>
                <field var="date" type="text-single">
                        <value>1374144203775</value>
                </field>
        </x>
<active xmlns="http://jabber.org/protocol/chatstates"/>
</message>

REQUEST:

<iq id='ala7' type='get'>
                <retrieve xmlns='urn:xmpp:archive' start='2014-03-31T12:45:55.000+0000' with='000000000131@localhost'>
                                <set xmlns='http://jabber.org/protocol/rsm'>
                                                <max>8</max>
                </set>
        </retrieve>
</iq>

RESPONSE:

<iq type='result' to='000000000101@localhost/localhost' id='ala7'>
    <chat xmlns='urn:xmpp:archive' start='2014-03-31T12:45:55+0000' with='000000000131@localhost'>
        <from secs='4982'>
            <body>dfdfsfd</body>
        </from>
        <from secs='5060'>
            <body>aaaaaabbbbbbbbcccccccc</body>
        </from>
        <set xmlns='http://jabber.org/protocol/rsm'>
            <first index='0'>4982</first>
            <last>5060</last>
            <count>2</count>
        </set>
    </chat>
</iq>

Unfortunately additional fields were not present in response :( I used 5.3.0-SNAPSHOT b3491 2014-04-01 version.

Added by Andrzej Wójcik IoT 1 CloudTigaseTeam over 4 years ago

Please verify that you properly upgraded installation to snapshot version. I just used tigase-message-archiving.jar from a build you mentioned and I received correct results. I see that you still are receiving wrong first, last elements in response - which suggests that something went wrong during upgrade - now we ARE NOT using SRCS as IDs of a message.

How you upgraded this installation?

Did you changed all files or just a single file?

Did you used files from -dist archive or from -dist-max archive?

Added by Łukasz Soszyński over 4 years ago

I use maven dependency:

            <xmpp.sever.version>5.3.0-SNAPSHOT</xmpp.sever.version>

                        <dependency>
                <groupId>tigase</groupId>
                <artifactId>tigase-server-dist</artifactId>
                <classifier>dist-max</classifier>
                <version>${xmpp.sever.version}</version>
                <scope>provided</scope>
                <type>zip</type>
            </dependency>

from repository:

        <repository>
            <id>tigase-repository</id>
            <name>Tigase Repository</name>
            <url>http://maven.tigase.org</url>
        </repository>

After unpacking file described by the above maven dependency I replace init.properties and tigase.conf with my custom files.

Is this approach correct?

Added by Wojciech Kapcia TigaseTeam over 4 years ago

OK, while the approach is definitely interesting i have one but :-)

<xmpp.sever.version>5.3.0-SNAPSHOT</xmpp.sever.version>

and if you browse <url>http://maven.tigase.org</url> you will notice that this repository is only for final/stable releases. All snapshot dependencies reside on our build machine hence propper setup of the repositories would be:

    <repositories>
        <repository>
            <snapshots />
            <id>tigase</id>
            <name>Tigase repository</name>
            <url>http://maven.tigase.org</url>
        </repository>
        <repository>
            <id>tigase-snapshot</id>
            <name>Tigase repository</name>
            <url>http://build.xmpp-test.net/maven/</url>
            <snapshots>
                <enabled>true</enabled>
            </snapshots>
        </repository>
    </repositories>

Added by Łukasz Soszyński over 4 years ago

After upgrade in proper way everything works OK :) Thanks

Added by Łukasz Soszyński over 4 years ago

After upgrade in proper way everything works OK :) Thanks

    (1-9/9)