Project

General

Profile

Best Strategy for Timeouts

Luca Stucchi
Added about 4 years ago

Hi there,

I am implementing a timeout strategy, whose meaning is "If a message wasn't delivered by X seconds, then add a tag to the original message and send a notification message".

I am thinking about implementing this in a component using addTimerTask(TimerTask task, long delay, TimeUnit unit);

Is this the correct way to do it, setting a timer for each received message ?

What about stopping one of those timers ? Imagine this operation should trigger in 15 seconds, but after 5 I can deliver the message, how can I get the TimerTask and call a .cancel() method because I no longer need that operation to be executed ?

Thanks in advance,

Luca


Replies (8)

Added by Wojciech Kapcia TigaseTeam about 4 years ago

It's not exactly what you are asking but wouldn't XEP-0079: Advanced Message Processing fit your requirement?

Added by Luca Stucchi about 4 years ago

Hi Wojciech,

I know and use XEP-0079, but as you can see on chapter 3.5.2 Expire-at Rules, using it only grants

error    | Return an error if the message would be delivered after the specified timestamp.

What I am looking for, instead, is an action triggered synchronously exactly at the expire time in case of non-delivered message.

And since I hopefully won't reach the expire time for the majority of the messages, I need to be able to cancel the TimerTask in order to block the expire action execution.

If it was possible to twist the XEP-0079 in order to implement this behavior I would be happy, but I fear it is just different from what the specification says.

Have you ever implemented or seen something similar ? What would you do if you had to implement this ?

Thanks in advance,

Luca

Avatar?id=6023&size=32x32

Added by Artur Hefczyc TigaseTeam about 4 years ago

The main questions are:

  1. Why do you want/need this feature?

  2. Mainly how critical this is to you and how much tolerance you can have?

  3. Are you sure you want this to be implemented on the servers side, not on the client side?

There are XEPs for message acknowledgments and delivery receipts. Maybe this would be enough.

I am giving you this considerations because tracking every single message in the server is extremely difficult. Imagine, we have installations with 500k messages/second being processed. That's half a million per second. Now, how long do you have your timeout to be? 10 seconds? 1 minute? For 10 secs timeout and with the traffic mentioned above, you would be tracking 5 million messages simultaneously. With 1 minute timeout 30 million messages. I am not implying that this is impossible. It is possible but it is also expensive as you need much more resources (servers) for a system which has to handle such a job.

Clients on the other hand, usually have very little to do and adding some more work for them put a minimal impact on the client. This is assuming you control the client code and can modify it.

Of course, if traffic on your installation is much lower, like 100 messages per second, then it is much easier to handle.

Added by Luca Stucchi about 4 years ago

Hi Artur,

Here are my answers for your questions:

  1. I need it because I have business logic to be activated on timeouts

  2. definitely critical, I mean, if the message can't be delivered, it's not a problem at all, but I need to be aware of that to trigger a reaction in the system. This is a requirement we can't relax. If with tolerance you mean if it needs to be precise to the microsec, no, but of course I have to rely on this feature, I just can't have the check performed 10 minutes later, that would be totally pointless !

  3. yes I am sure, because that's exactly the scenario I want to identify, a non-connected client which can't confirm the reception. After a timeout I want the system to be aware I can't deliver it, and activate some "reparation" logic. If I shift this on client side, I fall back in XEP-0079, meaning that when the client reconnects says "Oh, I read the message, but it expired". When ? Could be one second after the timeout, 10 seconds after, the day after... that's not the information I need.

I have complete access and control on the client (we implement it) but the scenario is different and we need something server-side.

BTW, I was looking at the code and I found out a nice inner class that looks very close to what I would like to implement : PacketReceiverTask in tigase.server.AbstractMessageReceiver

In particular, I was thinking about

  1. intercept a XEP-0079 packet with a plugin

  2. forward it to a component

  3. use addOutPacketWithTimeout with the original packet and implementing my reaction logic in the handler.

Does it make sense to you ? Since this is a requirement, I am aware that the system load and complexity will grow up, but that is not a problem

Thanks in advance,

Luca

Avatar?id=6023&size=32x32

Added by Artur Hefczyc TigaseTeam about 4 years ago

Ok, thank you for additional information. Now I have a better picture of the whole thing. One of our customers has implemented similar system for strict QoS/QA. They make payment over XMPP so they need to be 100% sure that every single message is delivered.

My suggestion to you would be to implement a similar thing based on the AMP implementation in Tigase. If you use AMP than you will tremendously reduce amount of work you require and you will preserve a lot of flexibility. Tigase's AMP works quite well and is optimized for a high traffic. Plus, what's even more important, it is implemented in plugin-component architecture which allows you to offload the hard work on a separate server.

If you look closely into the AMP specification and Tigase implementation you can see that AMP has actions and conditions. In Tigase these are plugins, so you can easily add more actions and conditions to plug your own logic and handlers. Even now, there are actions and conditions which can deal with messages not delivered in specified time. It does not fulfill all your requirements but it's a good start.

And then you have this great flexibility as you can more or less control the how message is handled from a client side by putting AMP payload properties.

Added by Luca Stucchi about 4 years ago

Hi Artur and thanks for the advise.

Yes, I will rely upon AMP and, basing on messages that will go back and forth, will intercept them and react.

I gave a glance to Tigase AMP implementation and I decided that it's not too wise to interfere: I will define custom plugins and components to implement the logic that I need outside for what can't be done with AMP.

Very interesting the part of offloading the hard work to a separate server, could you please direct me to a documentation page that deepen this question ?

Thanks again,

Luca

Avatar?id=6023&size=32x32

Added by Artur Hefczyc TigaseTeam about 4 years ago

Very interesting the part of offloading the hard work to a separate server, could you please direct me to a documentation page that deepen this question ?

That's a core of the Tigase API for components. If you write a component using Tigase API, you do not have to think or worry how it is deployed. It can be deployed as an internal Tigase component or as an external component. This is transparent from development point of view as this is just the server configuration option. It is described in the development guide describing components development and in an admin guide describing deployment of external components.

As for the "interfering" with AMP code. It is designed to be extensible. If you extend it by adding more plugins (actions and conditions) you are not interfering. You just add... more actions and conditions which would be triggered by a different AMP payload. So the existing logic would not be affected.

Added by Luca Stucchi about 4 years ago

Hi Artur,

I followed your indication for the extension: that enhances the readability and uniforms the protocol used. Looks like the smartest solution.

For the external component, by now I'll go on for a while using an internal one, but I see your point and I think in future I may be tempted to use external components.

Thanks again,

Luca

    (1-8/8)