Project

General

Profile

conection failed

Raylin lin
Added over 4 years ago

hi,all:

  I use tigase server(5.3 built by myself),and recently I found that the clients(spark or asmack) will lose connection when  the server have started up for a few days. and there are only less than 20,000 users(with less than 500 online users),and the machine configuration is very good. If i restart the tigase server, these clients will succeed to be connected again.

etc/init file:

[root@VMCHENGXIN etc]# cat init.properties
config-type=--gen-config-def
--admins=admin@www.mydomain.com
--virt-hosts = www.mydomain.com
--debug=server,xmpp,db
--user-db=mysql
--user-db-uri=jdbc:mysql://10.63.10.46:3306/tigase?user=admin&password=mypassword&noAccessToProcedureBodies=true&useUnicode=true&characterEncoding=utf-8
#--cluster-connect-all=true
#--cluster-mode=true
#--cl-conn-repo-class=tigase.cluster.repo.ClConConfigRepository
--cross-domain-policy-file=/etc/tigase/cross-domain-policy.xml
--bosh-extra-headers-file=/etc/tigase/bosh-extra-headers-file.txt
--bosh-ports=5280
--sm-plugins=+jabber:iq:storeLastMessage
sess-man/plugins-conf/amp/store-limit=1200
amp/store-limit[S]=1200
muc/muc-lock-new-room[B]=false
--comp-class-1 = tigase.muc.MUCComponent
--comp-name-1 = muc
--comp-name-2=pubsub
--comp-class-2=tigase.pubsub.PubSubComponent
#--comp-name-3 = ext  
#--comp-class-3 = tigase.server.ext.ComponentProtocol
--monitoring=jmx:9050,http:9080,snmp:9060
#--monitoring=http:9080

the log looks like this:

2014-09-30 10:31:55.204 [pool-3-thread-1]  ClientConnectionManager$StoppedHandler.timeOutExpired()  INFO: No response within time limit received for a packet: from=null, to=null, DATA=<iq to="sess-man@vmchengxin" from="c2s@vmchengxin/10.143.76.190_5222_119.62.127.131_27675" id="4f688c91-e61b-4796-a22d-0e214cbb7339" type="set"><command xmlns="http://jabber.org/protocol/commands" node="STREAM_CLOSED"/></iq>, SIZE=224, XMLNS=null, PRIORITY=SYSTEM, PERMISSION=NONE, TYPE=set
2014-09-30 10:31:55.204 [in_12-message-router]  MessageRouter.processPacket()  FINEST:  Processing packet: from=null, to=null, DATA=<iq to="sess-man@vmchengxin" from="c2s@vmchengxin/10.143.76.190_5222_1.85.17.90_48866" id="f42a2317-747f-41b0-9e7e-1e7e47d214dc" type="set"><command xmlns="http://jabber.org/protocol/commands" node="STREAM_CLOSED"/></iq>, SIZE=220, XMLNS=null, PRIORITY=SYSTEM, PERMISSION=NONE, TYPE=set
2014-09-30 10:31:55.206 [in_12-message-router]  MessageRouter.getLocalComponent()  FINEST: Called for : sess-man@vmchengxin
2014-09-30 10:31:55.206 [in_12-message-router]  MessageRouter.getLocalComponent()  FINEST: Called for : sess-man@vmchengxin
2014-09-30 10:31:55.206 [in_12-message-router]  MessageRouter.processPacket()  FINEST:  1. Packet will be processed by: sess-man@vmchengxin, from=null, to=null, DATA=<iq to="sess-man@vmchengxin" from="c2s@vmchengxin/10.143.76.190_5222_1.85.17.90_48866" id="f42a2317-747f-41b0-9e7e-1e7e47d214dc" type="set"><command xmlns="http://jabber.org/protocol/commands" node="STREAM_CLOSED"/></iq>, SIZE=220, XMLNS=null, PRIORITY=SYSTEM, PERMISSION=NONE, TYPE=set
2014-09-30 10:31:55.204 [pool-3-thread-1]  ClientConnectionManager$StoppedHandler.timeOutExpired()  INFO: No response within time limit received for a packet: from=null, to=null, DATA=<iq to="sess-man@vmchengxin" from="c2s@vmchengxin/10.143.76.190_5222_1.85.17.90_41635" id="7b6599c6-06fe-4860-85cc-80f0038f8725" type="set"><command xmlns="http://jabber.org/protocol/commands" node="STREAM_CLOSED"/></iq>, SIZE=220, XMLNS=null, PRIORITY=SYSTEM, PERMISSION=NONE, TYPE=set
2014-09-30 10:31:55.206 [in_12-message-router]  MessageRouter.processPacket()  FINEST:  Processing packet: from=null, to=null, DATA=<iq to="sess-man@vmchengxin" from="c2s@vmchengxin/10.143.76.190_5222_119.62.127.131_27675" id="4f688c91-e61b-4796-a22d-0e214cbb7339" type="set"><command xmlns="http://jabber.org/protocol/commands" node="STREAM_CLOSED"/></iq>, SIZE=224, XMLNS=null, PRIORITY=SYSTEM, PERMISSION=NONE, TYPE=set
2014-09-30 10:31:55.206 [in_12-message-router]  MessageRouter.getLocalComponent()  FINEST: Called for : sess-man@vmchengxin
2014-09-30 10:31:55.206 [in_12-message-router]  MessageRouter.getLocalComponent()  FINEST: Called for : sess-man@vmchengxin

Replies (9)

Avatar?id=6023&size=32x32

Added by Artur Hefczyc TigaseTeam over 4 years ago

When the spark or smack client looses connection can you reconnect without restarting the server? Are there any errors or warnings in the Tigase log file? Maybe you experience OOM?

Added by Raylin lin over 4 years ago

Without restarting the server,Neither spark nor smack client can be reconnected. There is no errors or warnnings in the tigase log file.maybe the error logs was washed away.

Avatar?id=6023&size=32x32

Added by Artur Hefczyc TigaseTeam over 4 years ago

Hm, if you have to restart Tigase server in order to reconnect clients it means a serious problem. Sounds like an OOM or some other critical condition. However, if there is no errors, warnings or anything in the ligase-console.log or any other log it is very hard to suggest something.

Do you have enough RAM? Do you have enough disk space? Maybe there are no logs because there is no disk space? Maybe this is a problem with database or connectivity with database? Is there enough diskspace on DB?

Added by Raylin lin over 4 years ago

Artur Hefczyc wrote:

Hm, if you have to restart Tigase server in order to reconnect clients it means a serious problem. Sounds like an OOM or some other critical condition. However, if there is no errors, warnings or anything in the ligase-console.log or any other log it is very hard to suggest something.

Do you have enough RAM? Do you have enough disk space? Maybe there are no logs because there is no disk space? Maybe this is a problem with database or connectivity with database? Is there enough diskspace on DB?

hi,Artur Hefczyc:

Found one Java-level deadlock:

=============================

presence Queue Worker 16

waiting to lock monitor 0x00007f2b00003688 (object 0x0000000783eac8f8, a tigase.xmpp.XMPPResourceConnection),

which is held by "jabber:iq:auth Queue Worker 10"

jabber:iq:auth Queue Worker 10

waiting to lock monitor 0x00007f2a6401e768 (object 0x0000000783eaad68, a tigase.xmpp.XMPPResourceConnection),

which is held by "jabber:iq:auth Queue Worker 0"

jabber:iq:auth Queue Worker 0

waiting to lock monitor 0x00007f2a8c0272b8 (object 0x0000000783eb8d08, a tigase.xmpp.XMPPResourceConnection),

which is held by "jabber:iq:auth Queue Worker 5"

jabber:iq:auth Queue Worker 5

waiting to lock monitor 0x00007f2a6401e768 (object 0x0000000783eaad68, a tigase.xmpp.XMPPResourceConnection),

which is held by "jabber:iq:auth Queue Worker 0"

Avatar?id=6023&size=32x32

Added by Artur Hefczyc TigaseTeam over 4 years ago

Ah, thread deadlock. We have seen this happen in 2 cases:

  1. When OpenJDK is used

  2. When certain version of mysql JDBC driver/library is used

I suggest you to use JDK from Oracle to run Tigase and to use JDBC driver from our code repository. Please let me know if that helps.

Added by Raylin lin over 4 years ago

hi,Artur Hefczyc:

thank you for your replies,which jdk version is suitable for tigase 5.3? my version is oracle jdk 1.7.0_51-b13.

[root@VMIMSERVER ~]# java -version

java version "1.7.0_51"

Java(TM) SE Runtime Environment (build 1.7.0_51-b13)

Java HotSpot(TM) 64-Bit Server VM (build 24.51-b03, mixed mode)

Added by Wojciech Kapcia TigaseTeam over 4 years ago

We are requiring jdk7 since version 5.3.0.

What version of mysql jdbc driver do you use? The one that has been bundled with the distribution archives?

Added by Raylin lin over 4 years ago

JarIndex-Version: 1.0

mysql-connector-java-5.1.11-bin.jar

com

com/mysql

com/mysql/jdbc

com/mysql/jdbc/configs

com/mysql/jdbc/exceptions

com/mysql/jdbc/exceptions/jdbc4

com/mysql/jdbc/integration

com/mysql/jdbc/integration/c3p0

com/mysql/jdbc/integration/jboss

com/mysql/jdbc/interceptors

com/mysql/jdbc/jdbc2

com/mysql/jdbc/jdbc2/optional

com/mysql/jdbc/log

com/mysql/jdbc/profiler

com/mysql/jdbc/util

org

org/gjt

org/gjt/mm

org/gjt/mm/mysql

Added by Wojciech Kapcia TigaseTeam about 4 years ago

Hm, in your client configuration do you use fixed bind resource name or allow server to configure it?

    (1-9/9)