Project

General

Profile

Tigase Scalable Architecture

Ildar Zaripov
Added over 1 year ago

I would like to continue the topic opened in the "Issues" https://projects.tigase.org/issues/6056

To my question "What sequence of components should the packet pass before it reaches the end user?", Wojciech Kapcia explained

The most important thing to understand is that Tigase is very modular and you can have multiple components running inside single instance. However one of the most important components is MessageRouter, which sits in the centre and serves as a, well, packet router directing them to the appropriate component.

There is also a group of specialised component responsible for handling users connections: ConnectionManagers (c2s, s2s, ws2s, bosh). They receive packet from the incoming connection, then they process then it goes to MessageRouter, which directs it to SessionManager component (with the session object referring to apropriate user in case of client to server connection, then the packet is processed and goes back to message router and then, based on the addressing can go to different component (muc, pubsub) or if it's addressed to another user it can go through

a) session manager (again), message router and then connection manager or,

b) server to server manager if the user or componet is on the different server

After his explanation I tried to draw a scheme of how I imagine the scalable Tigase server.

Can anyone comment on this scheme? In the picture below each block - is separate Tigase installation, running on, for example, Amazon EC2 instance.

What one should add/modify in the scheme so that you could use it in the real applications, with large message flow?


Replies (3)

Added by Wojciech Kapcia TigaseTeam over 1 year ago

This is not exactly correct:

  • inter-component communication happen over component protocol and not s2s (different ports and slightly different stream semantics (see https://xmpp.org/extensions/xep-0114.html and https://xmpp.org/extensions/xep-0225.html)

  • message-router wouldn't be a separate machine - it would be a part of each deployed instance (be that connection-manager or session-router - it's a core component and it's the first one being instantiated and responsible for routing messages internally)

  • MUC and PubSub - ideally you could/should use clustered versions of those components deployed on the same machines as session-manager, but if you decide to deploy them as external component then they would connect to session-manager machines.

Added by Ildar Zaripov over 1 year ago

Thanks for the reply!

  1. What is the difference between inter-component and s2s communication... I mean in what cases each of them are used?

  2. If I am not mistaken, plugins are responsible for stanza processing and they are controlled by the session manager. So, in order to handle many online users, one should have many session managers running on separate machines? (suppose, we are not going to work with pubsub, muc and etc.. only message and presence exchange)

I just want to understand how to build a scalable XMPP service, thanks in advance!

Avatar?id=6023&size=32x32

Added by Artur Hefczyc TigaseTeam over 1 year ago

Ildar Zaripov wrote:

Thanks for the reply!

  1. What is the difference between inter-component and s2s communication... I mean in what cases each of them are used?

s2s - server to server communication is used to exchange messages between different XMPP services, such as jabber.org, gtalk.com, tigase.im, and so on. These are different, separate installations with own users. In order to send messages between those users s2s protocol is used. This always takes place over TCP network connection.

inter-component communication in most cases takes place internally inside the Tigase XMPP Server instance, inside single JVM. It is used to exchange messages between different Tigase components: SM, MUC, PubSub, s2s, c2s, etc....

  1. If I am not mistaken, plugins are responsible for stanza processing and they are controlled by the session manager. So, in order to handle many online users, one should have many session managers running on separate machines? (suppose, we are not going to work with pubsub, muc and etc.. only message and presence exchange)

I just want to understand how to build a scalable XMPP service, thanks in advance!

The best way to build a scalable XMPP service based on Tigase is to run Tigase in cluster mode. That is entire Tigase XMPP server with all components and all plugins runs on many servers or VMs. This is called symmetric clustering. Each cluster node is a full Tigase XMPP Server with all functions. Scaling out means adding more cluster nodes, scaling down means removing cluster nodes. But changing number of cluster nodes does not affect overall functionality. Each node provides all the functions. Then, there is no single point of failure and it is actually easy to scale up and down and maintain the whole system.

    (1-3/3)