Project

General

Profile

Tigase clustering and carbon messages

Piotr Stolarski
Added about 5 years ago

We are trying to setup HA/LB tigase server instance, using tigase-server-5.2.0-beta3-b3269

We need carbon message support. With standard clustering if two resources of one user are logged to a different nodes

carbon message doesn't work. The only way we could overcome this issue was to run clustering in see-other-hosts mode.

The problem is that there is not many clients that supports see other hosts.

Is see-ether-hosts only option and we have to adapt our xmpp/client, or there is another solution to achieve HA an LB with tigase clustering + carbon ???


Replies (11)

Added by Wojciech Kapcia TigaseTeam about 5 years ago

There were a bit bug fixed regarding clustering between beta3 and rc1 - could you please your setup with the RC1 version?

Added by Piotr Stolarski about 5 years ago

thanks i will try to do it on RC1

Added by Piotr Stolarski about 5 years ago

We replaced beta3 with RC1. We use the same configuration as in beta 3, and we have issue with clustering.

When two users are logged into a different nodes massage dosn't reach endpoint, in area of one node it works correct.

Is this issue was reported before ??? or it is our local configuration problem?

Added by Wojciech Kapcia TigaseTeam about 5 years ago

Could you share your configuration? Are there any exceptions in the logs?

Added by Piotr Stolarski about 5 years ago

There are no exceptions in logs.

My config (2 nodes):

--cluster-mode = true

config-type = --gen-config-all

--debug = server,xmpp.impl,cluster

--user-db = mysql

--admins = admin@xmpp.xxxx.net

--user-db-uri = jdbc:mysql://10.32.13.104:3306/tigase?user=tigase_user&password=tigase_password&noAccessToProcedureBodies=true

--virt-hosts = xmpp.xxxx.net

#--cm-see-other-host=tigase.server.xmppclient.SeeOtherHostHashed

#c2s/cm-see-other-host/active=OPEN;LOGIN

--comp-name-1 = message-archive

--comp-class-1 = tigase.archive.MessageArchiveComponent

--sm-plugins = +message-archive-xep-0136

#,+sms-plugin,+wakeup,+stream-management-plugin,+message-carbons,-message,-msgoffline

#,-msgoffline,-amp,-message,+message-carbon,+jabber-iq-carbon,+sms-plugin,+wakeup

At this moment i'm sending message from user logged to one node to user on the second node, and i have no comunication.

I can see that messages are stored because after relogin messages appears from offline on the target user.

Added by Wojciech Kapcia TigaseTeam about 5 years ago

Are both nodes connected correctly? Do you see any entries in logs/tigase-console.log regarding password handshake for ClusterConnectionManager?

Added by Piotr Stolarski about 5 years ago

Correct .. I have found this in log ClusterConnectionManager.processHandshake() WARNING: Handshaking password doesn't match, disconnecting...

I'm using it on a fresh database .. what can i do to resolve this ??

Avatar?id=6023&size=32x32

Added by Artur Hefczyc TigaseTeam about 5 years ago

Looks like cluster nodes cannot connect to each other, hence communication between users connected to different cluster nodes is impossible. With your current setup the cluster should be in auto-configuration mode which requires DB access for the cluster code. There is still too little information to tell what is wrong:

  1. Could you please stop both nodes, remove all logs and start the system again

  2. Check the tigase-console.log it should print versions of the code used for the running service. This will ensure us whether you correctly updated your installation

  3. Make sure both cluster nodes connect to the same DB

  4. Could you provide us with a log file for analysis? The tigase-console.log and tigase.log.0?

  5. At least more data about the log warning, in some cases if the network configuration is incorrect the cluster node attempts to connect to itself

  6. You could also check the DB for cluster_nodes table. There should be an entry for each node. These entries are used to obtain passwords for inter-cluster connections. The entry also contains a hostname discovered by Tigase for the machine. Make sure this hostname matches what you expect for the machine and if the hostname is resolvable to a correct IP address.

  7. The WARNING entry in the log file should contain hostname information - please verify if it matches the DB entry

Added by Piotr Stolarski about 5 years ago

Ok thx for replay

My setup:

xmpp01.domain.net

xmpp02.domain.net

Both hosts have correct domain entries in etc/hosts and they can ping each other, using domain name.

In DB i removed entries in cluster_nodes table before restarting system.

After that 2 entries appeared

xmpp01.domain.net

xmpp02.domain.net

So i think that is correct.

Our config on both nodes i believe the same.

init.properties

--cluster-mode = true

config-type = --gen-config-all

--debug = server,xmpp.impl,cluster

--user-db = mysql

--admins = admin@xmpp.domain.net

--user-db-uri = jdbc:mysql://10.32.13.104:3306/tigase?user=tigase_user&password=tigase_password&noAccessToProcedureBodies=true

--virt-hosts = xmpp.domain.net

#--cm-see-other-host=tigase.server.xmppclient.SeeOtherHostHashed

#c2s/cm-see-other-host/active=OPEN;LOGIN

--comp-name-1 = message-archive

--comp-class-1 = tigase.archive.MessageArchiveComponent

--sm-plugins = +message-archive-xep-0136

#,+sms-plugin,+wakeup,+stream-management-plugin,+message-carbons,-message,-msgoffline

#,-msgoffline,-amp,-message,+message-carbon,+jabber-iq-carbon,+sms-plugin,+wakeup

tigase.conf

ENC="-Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8"

DRV="-Djdbc.drivers=com.mysql.jdbc.Driver:org.postgresql.Driver:org.apache.derby.jdbc.EmbeddedDriver"

#GC="-XX:+UseBiasedLocking -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:ParallelCMSThreads=2 -XX:-ReduceInitialCardMarks"

#EX="-XX:+OptimizeStringConcat -XX:+DoEscapeAnalysis -XX:+UseNUMA"

#JAVA_HOME="/opt/java/jdk1.7.0"

CLASSPATH=""

JAVA_OPTIONS="${GC} ${EX} ${ENC} ${DRV} -server -Xms100M -Xmx200M -XX:PermSize=32m -XX:MaxPermSize=256m -XX:MaxDirectMemorySize=128m "

TIGASE_CONFIG="etc/tigase.xml"

TIGASE_OPTIONS=" --property-file etc/init.properties "

Full logs from test in attached files

Avatar?id=6023&size=32x32

Added by Artur Hefczyc TigaseTeam about 5 years ago

Time difference between your nodes is too big and one of the node does not load an entry from another node as it "thinks" it is outdated. Hence it does not authenticate connection. Please make sure there is the same time on all machines. Once this is done, they should connect to each other.

Added by Piotr Stolarski about 5 years ago

Hi .. that was it, and carbon messages are working without --see-other-hosts, thank you very much.

    (1-11/11)