Project

General

Profile

Message Archiving Master/Slave Way?

Subir Jolly
Added over 5 years ago

I am planning on enabling "message-archive-xep-0136" so that I can start storing chat logs in my database. I have 3 options which I am looking at to have chat history in databases. The options are:

  • Master-Slave Database Configuration
  • The ability to direct READ requests to database slaves and WRITE requests to database masters.
  • Automatic Database Sharding(For Messages Only)
  • The ability to forward messages automatically different database servers based on user.
  • Store Subset Of Messages In Database
  • The ability to store a set of limited number of messages per user(i.e last 1000 messages..) in database.

Is any of the above mentioned options available in tigase? If not then is there a possibility of any of these being added to tigase?

Thanks,

Subir


Replies (4)

Avatar?id=6023&size=32x32

Added by Artur Hefczyc TigaseTeam over 5 years ago

Subir Jolly wrote:

I am planning on enabling "message-archive-xep-0136" so that I can start storing chat logs in my database. I have 3 options which I am looking at to have chat history in databases. The options are:

  • Master-Slave Database Configuration
  • The ability to direct READ requests to database slaves and WRITE requests to database masters.

You can setup DB for Tigase in master-slave mode but at the moment Tigase can use only one DB. So all reads and writes would be to the same DB. We are working on an option to distribute reads and writes. No ETA for this yet.

  • Automatic Database Sharding(For Messages Only)
  • The ability to forward messages automatically different database servers based on user.
  • Store Subset Of Messages In Database

This is available out of the box right now but only for different domains. So, basically, you can configure a different DB for each virtual domain.

Based on this, it is also possible to extend the logic to shard DBs based on other criteria, so sharing would be possible for users within a single virtual domain. But this option requires some customization.

  • The ability to store a set of limited number of messages per user(i.e last 1000 messages..) in database.

Is any of the above mentioned options available in tigase? If not then is there a possibility of any of these being added to tigase?

There is also a possibility to add more functions to the Tigase. You can either submit a features request and we will include it in our development plan or we can sign a development contract and you can sponsor development of a specific feature so it is available to you within a specific time-frame.

Added by Subir Jolly over 5 years ago

Thank you for the response.

Your work to distribute READs and WRITEs will definitely help a lot once it gets released.

As far as sharding by domain, we won't be able to use the default mechanism as we have implemented DynamicRostering for the same reason that we don't really know what all domains are going to be using this chat service. You did mention that we can customize the sharding logic. Which plugin would we have to implement/override and where can I find the documentation?

I would also submit a feature request for storing a subset of messages in database after a few days as I am not sure if it's going to be a feature request or a contract yet and it will depend on my project's timeline.

Thanks.

Avatar?id=6023&size=32x32

Added by Artur Hefczyc TigaseTeam over 5 years ago

Normally you would need to create own implementation of UserRepository and AuthRepository similar to corresponding UserRepositoryMD and AuthRepositoryMD. Unfortunately MessageArchive component does not follow this convention and does not use the same API, therefore there is no simple, configurable way to load own MessageRepository.

I suppose extending MessageArchiveDB and overriding methods which access DB to create a DB connection pool depending on some hashing would not be very complex. Unfortunately you would need to modify the component code to load own MessageArchiveDB instead of the default one.

Added by Subir Jolly over 5 years ago

Ah I see. Thank you so much for the response. If we plan on enabling chat history, then we will contact you if we want to set up a contract. Thanks again.

    (1-4/4)