Project

General

Profile

Connection timeout when using WebSockets

Claus Weng
Added almost 4 years ago

Hi

We are experiencing that if we remove the connection (for example by removing the ethernet cable) then the Tigase server still thinks that the client is online.

This is a problem for us because we need to know if a client is online or not.

Is there a way we configure a timeout for the WebSocket connection or get an instant update when the connection is lost.

We are using Tigase 5.2.2-b3463 together wit strophe.im 1.1.3 on the client side.

A NOTE: If we are using CORS then it works

Looking forward to hear from you.

Best regards

Claus Weng, Nordija


Replies (13)

Added by Wojciech Kapcia TigaseTeam almost 4 years ago

You can either modify system settings (v. TCP_keepalive) or use Tigase watchdog (v. --watchdog_timeout)

Avatar?id=6023&size=32x32

Added by Artur Hefczyc TigaseTeam almost 4 years ago

Claus, the problem you experience is related to the fact that the operating system does not report connection broken event to application when you remove ethernet cable. I think this is kind of OS level optimization and intentional logic. I tested this as well a while ago and to my surprise, when you disconnect ethernet cable for a few seconds and reconnect it afterwards, all the network connections continue to work as if you never removed the cable.

The time after which the OS reports connection broken to application depends on the OS and OS settings. As you cannot always control the OS level settings (especially on the client side) we suggest to setup Tigase watchdog to periodically test users' connections. However, this is kind of 2 edge sword, if you set XMPP ping time too short (very frequent pings) you can discover user disconnection very soon but then you also generate a lot of traffic between the client and the server and a significant load on the server. Imagine 100k connections tested every 10 seconds each - you get 10k pings per second in total.

Added by Claus Weng almost 4 years ago

Hi,

Thank you very much for the replies.

We do understand that we cannot set the watchdog too low and we will try to find a way to come around this issue.

Regards Claus

Added by Claus Weng almost 4 years ago

Hi,

Is it possible to get a kind of an acknowledge when you send a message. I mean can the client which sends the message get a confirmation that the recipient received the message?

I know I can implement that logic myself but I was wondering if the Tigase server has a functionality like that. (I would properly know if I knew more about the XMPP standard ;-)

Regards Claus

Avatar?id=6023&size=32x32

Added by Artur Hefczyc TigaseTeam almost 4 years ago

There are quite a few XEPs for this, each covers a different communication level:

  1. Stream Management

  2. Advanced Message Processing

  3. Message Delivery Receipts

All of them are supported by Tigase software, on both the server and the client side.

Added by Claus Weng almost 4 years ago

Hi again

Thank you very much for your help - I much appreciate it.

Best regards

Claus Weng

Added by Claus Weng over 3 years ago

Hi again

I've now had the chance to look into configuring the watchdog and implementing the ping/pong functionality on the clients.

For testing I've configured my server with:

ws2s/watchdog_ping_type[S]=xmpp

ws2s/watchdog_delay[L]=30000

ws2s/watchdog_timeout[L]=25000

As soon as the server hasen't seen any activity withing the last 25 seconds then the server makes a ping to the client.

When I remove the connection from one of my clients then I have different behaviour depending on the OS:

A) On my Mac I can see that the client will loose it's presence on the server after approximately 1 minute.

B) On our pre-production setup on Debian it takes a very long time before the client loose the presence. I just see the Tigase server continuously making ping requests to the client.

Is it possible for me to control how long Tigase should wait for the ping result before putting the client in not present?

Looking forward to hear from you

Regards Claus

Added by Wojciech Kapcia TigaseTeam over 3 years ago

Claus Weng wrote:

For testing I've configured my server with:

ws2s/watchdog_ping_type[S]=xmpp

ws2s/watchdog_delay[L]=30000

ws2s/watchdog_timeout[L]=25000

As soon as the server hasen't seen any activity withing the last 25 seconds then the server makes a ping to the client.

That is correct behaviour

When I remove the connection from one of my clients then I have different behaviour depending on the OS:

A) On my Mac I can see that the client will loose it's presence on the server after approximately 1 minute.

B) On our pre-production setup on Debian it takes a very long time before the client loose the presence. I just see the Tigase server continuously making ping requests to the client.

This should not happen. After failed sending ping to the client such connection should be terminated and subsequently unavailable presence should be broadcasted. Can you verify logs that the ping packet trigger failure of writing to socket?

Is it possible for me to control how long Tigase should wait for the ping result before putting the client in not present?

That's controlled by @watchdog_timeout@.

Please be aware, that above configuration applies only to WebSocket connections.

Added by Claus Weng over 3 years ago

Hi

I'm aware that I only configured for WebSocket.

These are the logs I keep getting (tail -f logs/tigase.log.0 | grep 'to="100666@tigase.nordija.dk/00:02:02:5b'):

2015-05-13 09:11:59.940 [Watchdog - ws2s]  ConnectionManager.writePacketToSocket()  FINEST: ws2s@tigase.nordija.dk/192.168.6.16_5290_192.168.4.24_1136, type: accept, Socket: ws2s@tigase.nordija.dk/192.168.6.16_5290_192.168.4.24_1136 Socket[addr=/192.168.4.24,port=1136,localport=5290], jid: 100666@tigase.nordija.dk/00:02:02:5b:57:6c, Writing packet: from=null, to=ws2s@tigase.nordija.dk/192.168.6.16_5290_192.168.4.24_1136, DATA=<iq id="tigase-ping-2501" from="tigase.nordija.dk" to="100666@tigase.nordija.dk/00:02:02:5b:57:6c" type="get"><ping xmlns="urn:xmpp:ping"/></iq>, SIZE=144, XMLNS=null, PRIORITY=NORMAL, PERMISSION=NONE, TYPE=get
2015-05-13 09:11:59.940 [Watchdog - ws2s]  WebSocketXMPPIOService.writeData()  FINEST:  sending data = <iq id="tigase-ping-2501" from="tigase.nordija.dk" to="100666@tigase.nordija.dk/00:02:02:5b:57:6c" type="get"><ping xmlns="urn:xmpp:ping"/></iq>
2015-05-13 09:12:39.941 [Watchdog - ws2s]  ConnectionManager.writePacketToSocket()  FINEST: ws2s@tigase.nordija.dk/192.168.6.16_5290_192.168.4.24_1136, type: accept, Socket: ws2s@tigase.nordija.dk/192.168.6.16_5290_192.168.4.24_1136 Socket[addr=/192.168.4.24,port=1136,localport=5290], jid: 100666@tigase.nordija.dk/00:02:02:5b:57:6c, Writing packet: from=null, to=ws2s@tigase.nordija.dk/192.168.6.16_5290_192.168.4.24_1136, DATA=<iq id="tigase-ping-2503" from="tigase.nordija.dk" to="100666@tigase.nordija.dk/00:02:02:5b:57:6c" type="get"><ping xmlns="urn:xmpp:ping"/></iq>, SIZE=144, XMLNS=null, PRIORITY=NORMAL, PERMISSION=NONE, TYPE=get
2015-05-13 09:12:39.941 [Watchdog - ws2s]  WebSocketXMPPIOService.writeData()  FINEST:  sending data = <iq id="tigase-ping-2503" from="tigase.nordija.dk" to="100666@tigase.nordija.dk/00:02:02:5b:57:6c" type="get"><ping xmlns="urn:xmpp:ping"/></iq>

"Can you verify logs that the ping packet trigger failure of writing to socket?"

  • How do I verify this exactly?

Added by Wojciech Kapcia TigaseTeam over 3 years ago

In principle writing to a broken socket should trigger and exception and subsequently removing broken connection from the collection.

Have you tried adjusting underlying OS settings, i.e. TCP_keepalive

Added by Claus Weng over 3 years ago

Hi again

I've been busy working on other stuff but now I'm back and testing the watchdog functionality.

I can see that changing the OS settings actually has an effect in the response time.

BUT I'm wondering why we cannot configure the timeout on the ping request from the tigase server.

Something like http://docs.oracle.com/javase/7/docs/api/java/net/Socket.html#setSoTimeout(int)

I'm looking forward to hear from you

Regards Claus

Added by Claus Weng over 3 years ago

By the way: Why use the watchdog at all if I still need to adjust in the OS?

Added by Wojciech Kapcia TigaseTeam over 3 years ago

In case on xmpp watchdog ping we don't verify whether client responded explicitly with the ping packet. The logic check whether the packet was successfully written to the socket (so if the connection is broken exception will be thrown) and then compare to the last time any XMPP packet was received. Please grep for all transmissions/events for "ws2s@tigase.nordija.dk/192.168.6.16_5290_192.168.4.24_1136" (in your above example), i.e. check CIDs (connection IDs instead of XMPP addresses).

    (1-13/13)