Project

General

Profile

Got a c2s/Socket Overflow, while in/out queue wait show zero values

Nauman Bashir
Added over 4 years ago

I am investigating an issue on our XMPP deployment. We have an occurrence of Socket Overflow in c2s component. c2s/out queue wait had zero packets.

I've checked the source code of SocketIO class, from where this variable is set. SocketIO line 416:

if (!dataToSend.offer(buff)) {

            ++buffOverflow;

            ++totalBuffOverflow;

        }

What I am not sure is, what are all the possible reasons of failure at this point: dataToSend.offer(buff)

Server specs:

Version: 5.2.0

OS: Amazon Linux AMI release 2014.03 (64 bit)

Heap : 7-12 GB

Please let me know if you require any additional information.

Thanks.


Replies (1)

Avatar?id=6023&size=32x32

Added by Artur Hefczyc TigaseTeam over 4 years ago

c2s component in/out queues are high level queues keeping XMPP packets for all connections. They can overflow if the overall traffic is so high that it cannot cope with distributing packets to individual connections. This does not happen though.

The other level of buffering, lower and closer to network layer is in the Socket object which queues buffers (binary data) for an individual connection. This can get overfilled when amount of data/traffic for individual connection is higher than it can be pushed. There are 2 possible reasons for this to happen:

  1. Settings for c2s connections are optimized for a low traffic per connection. This is to reduce resources usage per connection and normally works very well for a typical XMPP service as people usually do not generate high traffic on their connections. They do not send high number of messages per second.

  2. Most of Amazon EC2 VMs provide you with very poor I/O performance. You need to check the I/O throughput on your instance and try a different option with a higher I/O throughput. For high traffic installations we recommend EC2 instances with the highest I/O throughput that are offered.

    (1-1/1)