Project

General

Profile

Database connection issue

Anonymous
Added almost 4 years ago

Hi,

I have deployed server and database into 2 different servers (linux-based, software is the same but different IP addresses).

It was working fine initially, but after sometime, the database connection will be broken, with the following logs:

======================================================

2015-03-29 19:31:02.848 [Thread-3077]      ThreadExceptionHandler.uncaughtException()  SEVERE: Uncaught thread: "Thread-3077" exception
java.lang.IllegalAccessError: tried to access method tigase.server.AbstractMessageReceiver.addOutPacketWithTimeout(Ltigase/server/Packet;Ltigase/server/ReceiverTimeoutHandler;JLjava/util/concurrent/TimeUnit;)Z from class tigase.muc.Ghostbuster2
        at tigase.muc.Ghostbuster2.ping(Ghostbuster2.java:308)
        at tigase.muc.Ghostbuster2.ping(Ghostbuster2.java:285)
        at tigase.muc.MUCComponent$1.run(MUCComponent.java:243)
2015-03-29 20:14:06.354 [jabber:iq:register Queue Worker 0]  RepositoryAccess.register()  SEVERE: Repository access exception.
tigase.db.TigaseDBException: Problem accessing repository.
        at tigase.db.jdbc.TigaseCustomAuth.addUser(TigaseCustomAuth.java:362)
        at tigase.db.AuthRepositoryMDImpl.addUser(AuthRepositoryMDImpl.java:81)
        at tigase.xmpp.RepositoryAccess.register(RepositoryAccess.java:473)
        at tigase.xmpp.impl.JabberIqRegister.process(JabberIqRegister.java:239)
        at tigase.server.xmppsession.SessionManager$ProcessorWorkerThread.process(SessionManager.java:2689)
        at tigase.util.WorkerThread.run(WorkerThread.java:132)
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
        at sun.reflect.GeneratedConstructorAccessor61.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
        at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1119)
        at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2257)
        at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:784)
        at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46)
        at sun.reflect.GeneratedConstructorAccessor59.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
        at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:354)
        at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:284)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:270)
        at tigase.db.jdbc.DataRepositoryImpl.initRepo(DataRepositoryImpl.java:483)
        at tigase.db.jdbc.DataRepositoryImpl.checkConnection(DataRepositoryImpl.java:422)
        at tigase.db.jdbc.DataRepositoryImpl.getPreparedStatement(DataRepositoryImpl.java:209)
        at tigase.db.DataRepositoryPool.getPreparedStatement(DataRepositoryPool.java:179)
        at tigase.db.jdbc.TigaseCustomAuth.addUser(TigaseCustomAuth.java:346)
        at tigase.db.AuthRepositoryMDImpl.addUser(AuthRepositoryMDImpl.java:81)
        at tigase.xmpp.RepositoryAccess.register(RepositoryAccess.java:473)
        at tigase.xmpp.impl.JabberIqRegister.process(JabberIqRegister.java:239)
        at tigase.server.xmppsession.SessionManager$ProcessorWorkerThread.process(SessionManager.java:2689)
        at tigase.util.WorkerThread.run(WorkerThread.java:132)
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

======================================================================

I saw such message in console log when I wanted to restart:

===============================
Please upgrade database schema now.
Current scheme version is: null, expected: 5.1
Check the schema upgrade guide at the address:
http://www.tigase.org/content/tigase-51-database-schema-upgrade
----
If you have upgraded your schema and you are still
experiencing this problem please contact support at
e-mail address: support@tigase.org

===============================

and it cannot be started anymore, I will need to reset the tigase database (basically drop the database and setup again) in order to make it work, but database issue happened with all the error message above again.

May I know what's the issue?


Replies (8)

Added by Anonymous almost 4 years ago

Here is the server version: 5.2.1-b3461

Thank you!

Added by Wojciech Kapcia TigaseTeam almost 4 years ago

Please upgrade database schema now.
Current scheme version is: null, expected: 5.1

It looks like you have problem with you database as it either removes the data or corrupt it. After the deployment and creation of the schema there is information about schema version. This information is not overriten at any point. Please check if something else is not removing any data or schema information from the database. There is no such mechanism within Tigase itself.

Added by Anonymous almost 4 years ago

Hi Wojciech,

Thanks for your reply.

Noted about the corrupted data, but my concern here is, this only happens when I split MySQL database and Tigase into 2 different servers (be more precise, 2 different aws instances), when I put 2 components in one server, this is not happening anymore.

So my question here is, any specific requirement to split server and database? Because I can always replicate this data corruption by putting server and database apart.

Added by Wojciech Kapcia TigaseTeam almost 4 years ago

What do you mean by 'split database'?! You should use the same database instance for all Tigase nodes (cluster wise). If you want to have database failover then you need to configure MySQL replication correctly.

Added by Anonymous almost 4 years ago

Thanks for prompt reply.

What I meant was, splitting server and database by having 2 components in different servers, relying on network connection for tigase node to connect to MySQL database.

---------                ---------
|  Server   |  network |  Server  |
|  Tigase  | ------->|  MySQL  |
|     A       |    3306    |     B       |  
|              |                |              |
---------                ---------

Sorry if my drawing screws up, but I would like to find out the possibility of having tigase in server A and mysql that serves tigase in server B, because such infrastructure is breaking tigase and causing data corruption.

Added by Wojciech Kapcia TigaseTeam almost 4 years ago

Of course it's possible - it's only a matter of providing correct JDBC URI. Please note, that you should be able to establish connection to other machine even from command line (i.e. please enable remote connections from MySQL server).

Added by Anonymous almost 4 years ago

Yes, I know it's possible by having a JDBC URI in the init.properties.

But as I mentioned, it works at the beginning (that means remote connection is fine), but after sometime, it breaks, by showing all the exception in the tigase-console.log.

Added by Eric Dziewa almost 4 years ago

Your connection to the database is broken. Something has caused the connection to sever. You can verify this many different ways: try connecting to the db from the Tigase Server machine with mysql client @$mysql -h hostname.com -u username -p@. On the db host @$lsof -i -P |grep -i established@.

    (1-8/8)