Pending SM messages lost during shutdown

Daniele Ricci
Added almost 3 years ago

Hello project members,

I noticed that if messages are in Session Management (as in XEP-0198) queue, if the server shuts down and messages are still pending because of a network issue on the recipient side, those messages are lost forever. I had a couple of controlled cases, is that possible that those messages are not handled at all by design?

I'm using the offlinemsg plugin and not using AMP for storing messages. I know the offlinemsg plugin quite well and I know it deletes messages from the database before they are even delivered, so might it cause message loss on server shutdown?

Also, SM resumption is not enabled by my clients, but I don't know it makes a difference in this case (does it?)


Replies (2)

Added by Andrzej Wójcik IoT 1 CloudTigaseTeam over 2 years ago

Messages which are in processing queue and in session management queues may be lost during server shutdown. This is caused by the fact that currently shutdown is initiated termination signal being sent to server. This all is done by design to allow fast shutdown and later fast startup which is needed during server restart.

Yes, message loss can be caused by that, however message processing and delivery should be fast so it is rather hard to loss message due to that.

StreamManagement is responsible for queuing packets sent to client until they are confirmed as delivered. Right now this messages are kept in memory until connection is dropped, stream is resumed or packets are confirmed as delivered. So server shutdown may cause drop of this messages as well.

We will work on improvements to message delivery and graceful server shutdown in version 7.2.0. We already started by sending stream termination to clients when server will be stopped which will trigger message redelivery from stream resumption queue.

Added by Daniele Ricci over 2 years ago

Thanks for the insightful info! Looking forward to an even better shutdown handling.