Project

General

Profile

Tigase cluster basic info + ACS info

Andrew Beni
Added 12 months ago

Hi,
I'm running tigase 7.1.3 with a single instance and I want to migrate to the cluster, if I understood correctly there is no need of a license for the standard setup but only for ACS that is activated by

--sm-cluster-strategy-class=tigase.server.cluster.strategy.OnlineUsersCachingStrategy

If I don't use it I don't need a license. Correct?

this is my init.properties

--virt-hosts = {{salt['pillar.get']('tigase:host')}}
--vhost-disable-dns-check = true
--admins = {{salt['pillar.get']('tigase:admin_username')}}, {{salt['pillar.get']('tigase:admin2_username')}}

--domain-filter-policy = OWN
--vhost-register-enabled = false
--user-db = {{salt['pillar.get']('tigase:db')}}
--user-db-uri = jdbc:mysql://{{salt['pillar.get']('tigase:db_host')}}:{{salt['pillar.get']('tigase:db_port')}}/{{salt['pillar.get']('tigase:db_name')}}?user={{salt['pillar.get']('tigase:db_user')}}&password={{salt['pillar.get']('tigase:db_password')}}&useUnicode=true&characterEncoding=UTF-8&autoCreateUser=true


basic-conf/logging/java.util.logging.FileHandler.pattern = /var/log/tigase/tigase.log
basic-conf/logging/java.util.logging.FileHandler.limit=100000000
basic-conf/logging/java.util.logging.FileHandler.count=20

sess-man/plugins-conf/enabled-mechanisms=PLAIN,DIGEST-MD5
bosh/concurrent-requests[I] = 2
bosh/max-inactivity[L] = 10

############################ WEBSOCKET #################################
ws2s/connections/ports[i]=5290,5291
ws2s/connections/5291/socket=ssl
ws2s/connections/5291/type=accept
ws2s/processors[s]=urn:xmpp:sm:3

############################ THROTTLING ################################
--cm-traffic-throttling=xmpp:0:0:disc,bin:0:0:disc

########################## OTHER #################################
config-type = --gen-config-def
--cluster-mode = false

--monitoring=jmx:9050

# https://tigase.net/content/stream-management
c2s/processors[s]=urn:xmpp:sm:3

sess-man/plugins-conf/amp/store-limit=0
sess-man/plugins-conf/jabber\:iq\:roster/max_roster_size=10000
amp/store-limit[S]=0

vhost-man/domains-per-user-limit[I]=100000000

--sm-plugins=+amp,-message,-msgoffline


as read in the documentation the only thing I have to do is to set

--cluster-mode = true

1) What is the min numer of nodes? 3?
2) What ports are used for autodiscovery? 4250 and 3540 ?
3) How I set a password for all nodes for autodiscovery and cluster comunications, is needed?

4) cm-see-other-host is mandatory?

5) I'm running tigase on EC2 and I have a private and public zone for the hostname this means tha mydomain.com resolves with a private IP inside amazon and resolve with a public IP outside, I have to use tigase.server.xmppclient.SeeOtherHostDualIP ?

6) How is set the HA? I mean I have to put a load balancer before the cluster or no load balancer and I have to set in the DNS all the IPs of the cluster?

7) The doc say:
All basic components support clustering configuration, and some may be turned on or off.
which must be turned off?

I use only AMP and WEBSOCKET, are them supported in cluster conf?

I would like to add
http://docs.tigase.org/tigase-server/7.1.2/Administration_Guide/html_chunk/_tigase_http_api.html
Is it supported in cluster conf?

8) What are the difference in performance between ACS and not ACS? How I get an ACS license and how much does it costs?

thanks in advance for any response


Replies (5)

Added by Andrew Beni 12 months ago

REMOVED

Added by Wojciech Kapcia TigaseTeam 12 months ago

Andrew Beni wrote:

Hi,
I'm running tigase 7.1.3 with a single instance and I want to migrate to the cluster, if I understood correctly there is no need of a license for the standard setup but only for ACS that is activated by
If I don't use it I don't need a license. Correct?

This is correct.

as read in the documentation the only thing I have to do is to set

--cluster-mode = true

Correct.

1) What is the min numer of nodes? 3?

No minimum.

2) What ports are used for autodiscovery? 4250 and 3540 ?

Auto-discovery is done via shared database (all cluster nodes store the information in cluster_nodes table)

3) How I set a password for all nodes for autodiscovery and cluster comunications, is needed?

You can set it manually (legacy and not recommended, but still works - please see documentation with properties) but auto-discovery generates a random password for each node.

4) cm-see-other-host is mandatory?

No.

5) I'm running tigase on EC2 and I have a private and public zone for the hostname this means tha mydomain.com resolves with a private IP inside amazon and resolve with a public IP outside, I have to use tigase.server.xmppclient.SeeOtherHostDualIP ?

If you are using SeeOtherHost then you may use it, but the actual hostname returned would be resolved by the client.

6) How is set the HA? I mean I have to put a load balancer before the cluster or no load balancer and I have to set in the DNS all the IPs of the cluster?

Yes - those are two options:

  • DNS Round Robin (adding all entries)
  • LB (or ELB in case of Amazon)

7) The doc say:
All basic components support clustering configuration, and some may be turned on or off.
which must be turned off?

I use only AMP and WEBSOCKET, are them supported in cluster conf?

Yes, they support clustered configuration. PubSub and MUC are most notable cases where clustering is more tricky (and you would be better off with ACS-MUC/PubSub)

I would like to add
http://docs.tigase.org/tigase-server/7.1.2/Administration_Guide/html_chunk/_tigase_http_api.html
Is it supported in cluster conf?

Yes, HTTP-API is relatively cluster independent (from configuration point of view)

8) What are the difference in performance between ACS and not ACS?

This depends also to which component are you referring, please take a look at: https://tigase.tech/projects/acs/wiki for more information

How I get an ACS license and how much does it costs?

Your instance should generate an installation-id for you installation once you enable ACS component/clustering - you should contact us via https://tigase.net/contact to get more details about pricing.

Also: please note that you should correctly configure your installation, including hostnames (and FQDN) hostname.

Added by Andrew Beni 12 months ago

Hi, first of all thanks for the response

Wojciech Kapcia wrote:

5) I'm running tigase on EC2 and I have a private and public zone for the hostname this means tha mydomain.com resolves with a private IP inside amazon and resolve with a public IP outside, I have to use tigase.server.xmppclient.SeeOtherHostDualIP ?

If you are using SeeOtherHost then you may use it, but the actual hostname returned would be resolved by the client.

I have made some tests without SeeOtherHost, in session manager for the nodes connected to the cluster I get:
ip-172-31-28-90.ec2.internal
ip-172-31-9-90.ec2.internal

this are not resolvable as public IP, are this hostnames forwarded in anyway to the clients? For redirect or other? I didn't understand if a busy cluster node can send the client to another node, in this case which hostname it will use? This hostname?

this are the data of cluster_nodes

mysql> select * from cluster_nodes;
+------------------------------+------------------------------+------------------------------------------------------------------+---------------------+------+------------+-----------+
| hostname                     | secondary                    | password                                                         | last_update         | port | cpu_usage  | mem_usage |
+------------------------------+------------------------------+------------------------------------------------------------------+---------------------+------+------------+-----------+
| ip-172-31-28-90.ec2.internal | ip-172-31-28-90.ec2.internal | REMOVED | 2018-03-30 17:31:10 | 5277 |  3.8910506 |   4.30116 |
| ip-172-31-9-90.ec2.internal  | ip-172-31-9-90.ec2.internal  | REMOVED | 2018-03-30 17:31:09 | 5277 | 0.80906147 |  4.634392 |
+------------------------------+------------------------------+------------------------------------------------------------------+---------------------+------+------------+-----------+

6) How is set the HA? I mean I have to put a load balancer before the cluster or no load balancer and I have to set in the DNS all the IPs of the cluster?

Yes - those are two options:

  • DNS Round Robin (adding all entries)
  • LB (or ELB in case of Amazon)

ok, what do you think is the best solution? Do you know if is better the OLD ELB or the new one?

7) The doc say:
All basic components support clustering configuration, and some may be turned on or off.
which must be turned off?

I use only AMP and WEBSOCKET, are them supported in cluster conf?

Yes, they support clustered configuration. PubSub and MUC are most notable cases where clustering is more tricky (and you would be better off with ACS-MUC/PubSub)

Ok, I don't use PubSub and MUC

8) What are the difference in performance between ACS and not ACS?

This depends also to which component are you referring, please take a look at: https://tigase.tech/projects/acs/wiki for more information

ok I will look

How I get an ACS license and how much does it costs?

Your instance should generate an installation-id for you installation once you enable ACS component/clustering - you should contact us via https://tigase.net/contact to get more details about pricing.

Also: please note that you should correctly configure your installation, including hostnames (and FQDN) hostname.
ok

Is possible to know a user on which cluster node is connected?

I read this old post of tigase on AWS
https://stackoverflow.com/questions/8670234/scaling-tigase-xmpp-server-on-amazon-ec2

which are the best virtual machine to use? More RAM or more CPU?

in the future I will have (hopefully) a lots of clients connected: about 800k sending very few messages: about 30 messages/seconds in total

I think is useful to have more RAM, correct?

thanks again

Added by Wojciech Kapcia TigaseTeam 12 months ago

Andrew Beni wrote:

Hi, first of all thanks for the response

Wojciech Kapcia wrote:

5) I'm running tigase on EC2 and I have a private and public zone for the hostname this means tha mydomain.com resolves with a private IP inside amazon and resolve with a public IP outside, I have to use tigase.server.xmppclient.SeeOtherHostDualIP ?

If you are using SeeOtherHost then you may use it, but the actual hostname returned would be resolved by the client.

I have made some tests without SeeOtherHost, in session manager for the nodes connected to the cluster I get:
ip-172-31-28-90.ec2.internal
ip-172-31-9-90.ec2.internal

this are not resolvable as public IP, are this hostnames forwarded in anyway to the clients? For redirect or other? I didn't understand if a busy cluster node can send the client to another node, in this case which hostname it will use? This hostname?

this are the data of cluster_nodes

mysql> select * from cluster_nodes;
+------------------------------+------------------------------+------------------------------------------------------------------+---------------------+------+------------+-----------+
| hostname                     | secondary                    | password                                                         | last_update         | port | cpu_usage  | mem_usage |
+------------------------------+------------------------------+------------------------------------------------------------------+---------------------+------+------------+-----------+
| ip-172-31-28-90.ec2.internal | ip-172-31-28-90.ec2.internal | REMOVED | 2018-03-30 17:31:10 | 5277 |  3.8910506 |   4.30116 |
| ip-172-31-9-90.ec2.internal  | ip-172-31-9-90.ec2.internal  | REMOVED | 2018-03-30 17:31:09 | 5277 | 0.80906147 |  4.634392 |
+------------------------------+------------------------------+------------------------------------------------------------------+---------------------+------+------------+-----------+

In that case, if you are using internal hostnames then you have following options:

  • assign correct, FQDN hostnames that would resolve to internal IP locally and to external IP from internet;
  • use SeeOtherHostDualIP
  • disable internal load-balancing/redirections.

The name may be used also in Stream Management therefore it's recommended to use hostnames resolvable from internet.

6) How is set the HA? I mean I have to put a load balancer before the cluster or no load balancer and I have to set in the DNS all the IPs of the cluster?

Yes - those are two options:

  • DNS Round Robin (adding all entries)
  • LB (or ELB in case of Amazon)

ok, what do you think is the best solution? Do you know if is better the OLD ELB or the new one?

This depends on particular use-case. If you don't change your configuration a lot and use FQDN hostnames then going with DNS round robin is the easiest and more stright-forward solution.

Is possible to know a user on which cluster node is connected?

Yes, but only when you use ACS strategy.
As for the information itself - you can get it using admin ad-hoc commands (via xmpp or HTTP admin panel)

I read this old post of tigase on AWS
https://stackoverflow.com/questions/8670234/scaling-tigase-xmpp-server-on-amazon-ec2

which are the best virtual machine to use? More RAM or more CPU?
in the future I will have (hopefully) a lots of clients connected: about 800k sending very few messages: about 30 messages/seconds in total
I think is useful to have more RAM, correct?

Everything depends on particular use case, and always it's recommended to perform a Load Test on the estimated values. In your case more memory feels like a better choice (no processing, maintaining lot of connections).

Added by Andrew Beni 12 months ago

Wojciech Kapcia wrote:

Andrew Beni wrote:

Hi, first of all thanks for the response

Wojciech Kapcia wrote:

5) I'm running tigase on EC2 and I have a private and public zone for the hostname this means tha mydomain.com resolves with a private IP inside amazon and resolve with a public IP outside, I have to use tigase.server.xmppclient.SeeOtherHostDualIP ?

If you are using SeeOtherHost then you may use it, but the actual hostname returned would be resolved by the client.

I have made some tests without SeeOtherHost, in session manager for the nodes connected to the cluster I get:
ip-172-31-28-90.ec2.internal
ip-172-31-9-90.ec2.internal

this are not resolvable as public IP, are this hostnames forwarded in anyway to the clients? For redirect or other? I didn't understand if a busy cluster node can send the client to another node, in this case which hostname it will use? This hostname?

this are the data of cluster_nodes

mysql> select * from cluster_nodes;
+------------------------------+------------------------------+------------------------------------------------------------------+---------------------+------+------------+-----------+
| hostname                     | secondary                    | password                                                         | last_update         | port | cpu_usage  | mem_usage |
+------------------------------+------------------------------+------------------------------------------------------------------+---------------------+------+------------+-----------+
| ip-172-31-28-90.ec2.internal | ip-172-31-28-90.ec2.internal | REMOVED | 2018-03-30 17:31:10 | 5277 |  3.8910506 |   4.30116 |
| ip-172-31-9-90.ec2.internal  | ip-172-31-9-90.ec2.internal  | REMOVED | 2018-03-30 17:31:09 | 5277 | 0.80906147 |  4.634392 |
+------------------------------+------------------------------+------------------------------------------------------------------+---------------------+------+------------+-----------+

In that case, if you are using internal hostnames then you have following options:

  • assign correct, FQDN hostnames that would resolve to internal IP locally and to external IP from internet;
  • use SeeOtherHostDualIP
  • disable internal load-balancing/redirections.

The name may be used also in Stream Management therefore it's recommended to use hostnames resolvable from internet.

thanks I have disabled load balancing anyway I saw that using
--tigase-primary-address=
--tigase-secondary-address=
resolvse the problem

6) How is set the HA? I mean I have to put a load balancer before the cluster or no load balancer and I have to set in the DNS all the IPs of the cluster?

Yes - those are two options:

  • DNS Round Robin (adding all entries)
  • LB (or ELB in case of Amazon)

ok, what do you think is the best solution? Do you know if is better the OLD ELB or the new one?

This depends on particular use-case. If you don't change your configuration a lot and use FQDN hostnames then going with DNS round robin is the easiest and more stright-forward solution.

Ok, thanks for the hint

Is possible to know a user on which cluster node is connected?

Yes, but only when you use ACS strategy.
As for the information itself - you can get it using admin ad-hoc commands (via xmpp or HTTP admin panel)

ok

I read this old post of tigase on AWS
https://stackoverflow.com/questions/8670234/scaling-tigase-xmpp-server-on-amazon-ec2

which are the best virtual machine to use? More RAM or more CPU?
in the future I will have (hopefully) a lots of clients connected: about 800k sending very few messages: about 30 messages/seconds in total
I think is useful to have more RAM, correct?

Everything depends on particular use case, and always it's recommended to perform a Load Test on the estimated values. In your case more memory feels like a better choice (no processing, maintaining lot of connections).

Understood

thanks again

    (1-5/5)