Project

General

Profile

Bug #5450

Failing to initialize repository could force-stop server?

Added by Wojciech Kapcia TigaseTeam almost 2 years ago. Updated almost 2 years ago.

Status:
Closed
Priority:
Normal
Target version:
Start date:
Due date:
2017-05-02
% Done:

100%

Estimated time:
Database:
n/a
Applicable version:
git/master
Source Code Disclaimer:

Description

2017-04-28 17:03:23.028 [main]             Kernel.injectDependencies()             WARNING:  Can't inject dependency to bean default (class: class tigase.db.beans.AuthRepositoryMDPoolBean$AuthRepositoryConfigBean) unloading bean default
RootCause:
   -> java.lang.RuntimeException: Could not initialize tigase.db.jdbc.TigaseCustomAuth for name 'default'
      [tigase.db.beans.MDPoolConfigBean.setInstances(MDPoolConfigBean.java:169)]
      -> tigase.db.DBInitException: Problem initializing jdbc connection: jdbc:derby:/Users/wojtek/dev/tigase/tigase-testsuite/tigase_test_db;create=true
         [tigase.db.jdbc.TigaseCustomAuth.initRepository(TigaseCustomAuth.java:621)]
         -> java.sql.SQLSyntaxErrorException: No method was found that matched the method call tigase.db.derby.StoredProcedures.tigAccountStatus(java.lang.String, java.sql.ResultSet[]), tried all combinations of object and primitive types and any possible type conversion for any  parameters the method call may have. The method might exist but it is not public and/or static, or the parameter types are not method invocation convertible.
            [org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)]
            -> ERROR 42X50: No method was found that matched the method call tigase.db.derby.StoredProcedures.tigAccountStatus(java.lang.String, java.sql.ResultSet[]), tried all combinations of object and primitive types and any possible type conversion for any  parameters the method call may have. The method might exist but it is not public and/or static, or the parameter types are not method invocation convertible.
               [org.apache.derby.iapi.error.StandardException.newException(Unknown Source)]
…
2017-04-28 17:03:49.813 [jabber:iq:register Queue Worker 6]  WorkerThread.run()    SEVERE:   tigase.server.xmppsession.SessionManager$ProcessorWorkerThread,(jabber:iq:register Queue Worker 6) Exception during packet processing: from=c2s@atlantiscity.local/127.0.0.1_5222_127.0.0.1_52400, to=sess-man@atlantiscity.local, DATA=<iq type="set" id="reg2" xmlns="jabber:client"><query xmlns="jabber:iq:register"><username>admin</username><password>stats</password><email>test_user@localhost</email></query></iq>, SIZE=180, XMLNS=jabber:client, PRIORITY=NORMAL, PERMISSION=LOCAL, TYPE=set
java.lang.NullPointerException
    at tigase.db.jdbc.TigaseCustomAuth.addUser(TigaseCustomAuth.java:346)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at tigase.stats.StatisticsInvocationHandler.invoke(StatisticsInvocationHandler.java:80)
    at com.sun.proxy.$Proxy31.addUser(Unknown Source)
    at tigase.db.AuthRepositoryMDImpl.addUser(AuthRepositoryMDImpl.java:66)
    at tigase.xmpp.impl.JabberIqRegister.createAccount(JabberIqRegister.java:161)
    at tigase.xmpp.impl.JabberIqRegister.doRegisterNewAccount(JabberIqRegister.java:335)
    at tigase.xmpp.impl.JabberIqRegister.process(JabberIqRegister.java:642)
    at tigase.server.xmppsession.SessionManager$ProcessorWorkerThread.process(SessionManager.java:2442)
    at tigase.util.WorkerThread.run(WorkerThread.java:128)

We already have a couple of cases where failing to initialise repository stops the server with an error -- should we consider above as well?

Associated revisions

Revision 7b560498 (diff)
Added by Andrzej Wójcik IoT 1 CloudTigaseTeam almost 2 years ago

#5450: fixed issue with exception thrown during initialization of repository not stopping it's instance from being used

Revision af6a618c (diff)
Added by Andrzej Wójcik IoT 1 CloudTigaseTeam almost 2 years ago

#5450: fixed issue with setup mode not working

History

#1 Updated by Andrzej Wójcik IoT 1 CloudTigaseTeam almost 2 years ago

There are two cases here:

  • failure to connect to default data source (default repository)

Without this many components and features may fail to work properly (including clustering) so we should stop a server.

  • failure to connect to other data sources

This may return some errors but I think that it should work.

I'm going to review code responsible for initialization of repositories as it may be handy to start a server before DB is started (or while it is starting) and connect to a database as it will become available.

#2 Updated by Andrzej Wójcik IoT 1 CloudTigaseTeam almost 2 years ago

  • Status changed from New to In QA
  • Assignee changed from Andrzej Wójcik to Wojciech Kapcia

I've found that even though the instance of a repository is not initialized properly it is injected to repository pool which leads to NPE during runtime. I made changes in a kernel to make sure it will not happen again. This should fix above NPE.

Moreover, I've added check during bootstrapping of Tigase XMPP Server to make sure that data sources are initialized before we will start loading components - to make it fail-fast with fewer errors being logged.

Please verify that this works for you. We cannot stop server at this point as it may happen that this exception will be thrown during server reconfiguration and stopping server at this point could be very inconvenient.

#3 Updated by Wojciech Kapcia TigaseTeam almost 2 years ago

  • Status changed from In QA to Feedback
  • Assignee changed from Wojciech Kapcia to Andrzej Wójcik

Andrzej Wójcik wrote:

There are two cases here:

  • failure to connect to default data source (default repository)

Without this many components and features may fail to work properly (including clustering) so we should stop a server.

+1

  • failure to connect to other data sources

This may return some errors but I think that it should work.

I'm not sure about that - it leaves the server in rather half-working state (and I was under the impression that you are in general more in favour of failing that allowing that, for example our discussion in DSL where we consider configuration "bad" if any of the items are wrong instead of ignoring them)

I'm going to review code responsible for initialization of repositories as it may be handy to start a server before DB is started (or while it is starting) and connect to a database as it will become available.

Andrzej Wójcik wrote:

I've found that even though the instance of a repository is not initialized properly it is injected to repository pool which leads to NPE during runtime. I made changes in a kernel to make sure it will not happen again. This should fix above NPE.

Moreover, I've added check during bootstrapping of Tigase XMPP Server to make sure that data sources are initialized before we will start loading components - to make it fail-fast with fewer errors being logged.

Please verify that this works for you.

It works for me, however the is a small but - 'config-type' = 'setup' results in stopping the server:

==========
STARTED Tigase Tue May  2 13:10:02 -03 2017 using:
    ./scripts/tigase.sh start etc/tigase.conf 
==========
Picked up JAVA_TOOL_OPTIONS: -Djava.awt.headless=true
componentInfo{Title=Tigase XMPP Server, Version=7.2.0-SNAPSHOT-b4809/7b560498 (2017-05-01/23:10:40), Class=tigase.xml.XMLUtils}
componentInfo{Title=Tigase XMPP Server, Version=7.2.0-SNAPSHOT-b4809/7b560498 (2017-05-01/23:10:40), Class=tigase.util.ClassUtil}
componentInfo{Title=Tigase XMPP Server, Version=7.2.0-SNAPSHOT-b4809/7b560498 (2017-05-01/23:10:40), Class=tigase.server.XMPPServer}
2017-05-02 13:10:02.390 [main]             ConfiguratorAbstract.parseArgs()        CONFIG:   Setting defaults: --property-file = etc/init.properties
2017-05-02 13:10:02.486 [main]             ConfigHolder.loadConfiguration()        CONFIG:   Loaded configuration:
--test = false
'config-type' = 'setup'
http () {
    setup () {
        'admin-password' = 'tigase'
        'admin-user' = 'admin'
    }
}





  =============================================================================
  ERROR! Terminating the server process.
  Problem initializing the server: tigase.kernel.KernelException: Can't find bean implementing class tigase.db.beans.DataSourceBean
  Please fix the problem and start the server again.
  =============================================================================





ShutdownThread started...

Total number of threads: 5
No locked threads.

Save thread-dump to file: logs/thread-dump.log, size: 1689
ShutdownThread finished...

We cannot stop server at this point as it may happen that this exception will be thrown during server reconfiguration and stopping server at this point could be very inconvenient.

Agreed, and before we considered the same:

  • if the server couldn't establish connection to repository during startup = bad state;

  • exception accessing repository after correct startup = ok state (temporary network problem for example that could be recovered).

#4 Updated by Andrzej Wójcik IoT 1 CloudTigaseTeam almost 2 years ago

  • Status changed from Feedback to In QA
  • Assignee changed from Andrzej Wójcik to Wojciech Kapcia

Wojciech Kapcia wrote:

Andrzej Wójcik wrote:

There are two cases here:

  • failure to connect to default data source (default repository)

Without this many components and features may fail to work properly (including clustering) so we should stop a server.

+1

  • failure to connect to other data sources

This may return some errors but I think that it should work.

I'm not sure about that - it leaves the server in rather half-working state (and I was under the impression that you are in general more in favour of failing that allowing that, for example our discussion in DSL where we consider configuration "bad" if any of the items are wrong instead of ignoring them)

Yes, you are right. I was just thinking about a possible use case.

I'm going to review code responsible for initialization of repositories as it may be handy to start a server before DB is started (or while it is starting) and connect to a database as it will become available.

Andrzej Wójcik wrote:

I've found that even though the instance of a repository is not initialized properly it is injected to repository pool which leads to NPE during runtime. I made changes in a kernel to make sure it will not happen again. This should fix above NPE.

Moreover, I've added check during bootstrapping of Tigase XMPP Server to make sure that data sources are initialized before we will start loading components - to make it fail-fast with fewer errors being logged.

Please verify that this works for you.

It works for me, however the is a small but - 'config-type' = 'setup' results in stopping the server:

[...]

It is already fixed. I've fixed this issue today so it is not part of latest snapshot build.

We cannot stop server at this point as it may happen that this exception will be thrown during server reconfiguration and stopping server at this point could be very inconvenient.

Agreed, and before we considered the same:

  • if the server couldn't establish connection to repository during startup = bad state;

  • exception accessing repository after correct startup = ok state (temporary network problem for example that could be recovered).

Yes, this is how it works now.

#5 Updated by Wojciech Kapcia TigaseTeam almost 2 years ago

  • Status changed from In QA to Closed
  • % Done changed from 0 to 100

Works OK now.

Also available in: Atom PDF