Project

General

Profile

Clustering tests

Avatar?id=6023&size=32x32

Artur Hefczyc TigaseTeam
Added almost 5 years ago

The whole Tigase server development is oriented around automatic tests we run using Tigase Test Suite. Our aim is to not release new stuff before it is tested using automated tests. Only this way we can be sure all the server features work the same way for every release. The drawback is that we first need to create tests, then we can run them to test the server. It is a bit like double development effort.

The suite evolves together with the server. At the beginning I was running only functional tests executing test cases sequentially. Then I added performance tests which required running many tests concurrently on a separate connections and stability tests which in turn required lower resources consumption and..... stability. Every new addition required slightly new features set.

A few months back when I was running load tests that was another challenge to run the test suite from multiple machines and again I had to lower resources consumption to be able to open more than 60k user connections from each test suite instance.

We extensively use Bosh in a few Tigase installations so to provide decent support for web clients I have added Bosh tests which required a different way to maintain connections to the server because the Bosh client can send each request on a separate TCP/IP connection.

Now the clustering in Tigase. We have working clustering mode in the server for a few months already but we decided not to release a new version before it is well tested using automatic tests. New challenge - open multiple connections from the test suite and each connection must end up on a different cluster node.

Today I am happy to announce all the cluster tests have passed. This is a big step for us in our preparation for publishing Tigase 4.0. Now we can focus on those countless tiny bits required to prepare binary packages for the new version.

Below is a copy of the clustering tests report, for those who are interested in more details.

The tests have been run on the cluster installation with 3 nodes in domain: xmpp-test.tigase.org. If you look carefully you may notice uneven times for some tests depending on the node used. This is because we don't have 3 identical machines to do our tests. One of the node is QuadCore quite recent box while others are an old IBM laptop and 7 years old P4. Therefore these tests results can not be used as the performance indication. They are functional tests for the Tigase server working in the cluster mode.

Server version info:

Name:

 

 

Tigase

Version:

 

 

4.0.0-b1182

OS:

 

 

Linux-amd64-2.6.23-gentoo-r8, Java HotSpot(TM) 64-Bit Server VM-10.0-b22-Sun Microsystems Inc.

Local IP:

 

 

81.143.149.100

Remote IP:

 

 

81.143.149.98

Server basic configuration parameters:

sess-man:

admins

 

kobit@xmpp-test.tigase.org

sess-man:

anonymous-domains

 

xmpp-test.tigase.org, test-domain

sess-man:

auth-repo-class

 

tigase.db.jdbc.DrupalAuth

sess-man:

auth-repo-url

 

jdbc:mysql://localhost/xmpp_test?user=xxxx&password=xxxxxx&online_status=true

sess-man:

component-id

 

sess-man@veloci.tigase.org

sess-man:

def-hostname

 

veloci.tigase.org

sess-man:

hostnames

 

xmpp-test.tigase.org, test-domain

sess-man:

max-queue-size

 

10000

sess-man:

plugins

 

jabber:iq:auth, urn:ietf:params:xml:ns:xmpp-sasl, urn:ietf:params:xml:ns:xmpp-bind, urn:ietf:params:xml:ns:xmpp-session, roster-presence, jabber:iq:privacy, msgoffline, jabber:iq:version, http://jabber.org/protocol/stats, starttls, vcard-temp, http://jabber.org/protocol/commands, jabber:iq:private, urn:xmpp:ping, http://jabber.org/protocol/pubsub

sess-man:

trusted

 

admin@xmpp-test.tigase.org, admin@test-domain

sess-man:

user-repo-class

 

tigase.db.jdbc.JDBCRepository

sess-man:

user-repo-url

 

jdbc:mysql://localhost/xmpp_test?user=drupal&password=drupal12&autoCreateUser=true

Tests results:

Test name

Result

Total time

OK

Average

Description

History

Version

success

0 sec

1

22 ms

Get server version

Version

Configuration

success

0 sec

1

56 ms

Server configuration

Configuration

Statistics

success

0 sec

1

48 ms

Server statistics

Statistics

Test 1

success

0 sec

3

7 ms

Socket connect to all cluster nodes.

Test 1

Test 2

success

0 sec

3

17 ms

Stream open to all cluster nodes.

Test 2

Test 3

success

1 sec

3

392 ms

TLS connection open to all cluster nodes.

Test 3

Test 4

success

0 sec

3

146 ms

SASL authorization to all cluster nodes to different accounts.

Test 4

Test 5

success

0 sec

3

121 ms

SASL authorization to all cluster nodes to the same account 'tester'.

Test 5

Test 6

success

0 sec

3

66 ms

Message send within one node to self user.

Test 6

Test 7

success

1 sec

3

215 ms

Message send within 'veloci.tigase.org' node from 'admin' and from 'all-xmpp-test_1'to 'tester' and wait for a response.

Test 7

Test 8

success

1 sec

3

259 ms

Message send within 'shell2.tigase.org' node from 'admin' and from 'all-xmpp-test_1'to 'tester' and wait for a response.

Test 8

Test 9

success

1 sec

3

226 ms

Message send within 'test-i.tigase.org' node from 'admin' and from 'all-xmpp-test_1'to 'tester' and wait for a response.

Test 9

Test 10

success

1 sec

3

262 ms

Message send beetwen 3 nodes: from 'shell2.tigase.org', user 'admin' and from 'test-i.tigase.org', user 'all-xmpp-test_1' to user 'tester-1' on 'veloci.tigase.org' and wait for a response.

Test 10

Test 11

success

1 sec

3

232 ms

Message send beetwen 3 nodes: from 'test-i.tigase.org', user 'admin' and from 'veloci.tigase.org', user 'all-xmpp-test_1' to user 'tester-2' on 'shell2.tigase.org' and wait for a response.

Test 11

Test 12

success

1 sec

3

264 ms

Message send beetwen 3 nodes: from 'veloci.tigase.org', user 'admin' and from 'shell2.tigase.org', user 'all-xmpp-test_1' to user 'tester-3' on 'test-i.tigase.org' and wait for a response.

Test 12

Test 13

success

0 sec

2

169 ms

2 user connections, each using a different resource on 'veloci.tigase.org' node.

Test 13

Test 14

success

0 sec

2

213 ms

2 user connections, each using a different resource on 'shell2.tigase.org' node.

Test 14

Test 15

success

0 sec

2

180 ms

2 user connections, each using a different resource on 'test-i.tigase.org' node.

Test 15

Test 16

success

1 sec

2

274 ms

2 user connections, on node 'veloci.tigase.org' and 'shell2.tigase.org'.

Test 16

Test 17

success

0 sec

2

241 ms

2 user connections, on node 'shell2.tigase.org' and 'test-i.tigase.org'.

Test 17

Test 18

success

0 sec

2

192 ms

2 user connections, on node 'veloci.tigase.org' and 'test-i.tigase.org'.

Test 18

Test 19

success

0 sec

3

19 ms

XEP-0092 get-version - 3 user connections, each using a different node.

Test 19

Test 20

success

1 sec

3

92 ms

XEP-0039 Statistics Gathering - 3 user connections, each using a different node.

Test 20

Test 21

success

1 sec

3

79 ms

Test for feature-not-implemented error - 3 user connections, each using a different node.

Test 21

Test 22

success

0 sec

3

113 ms

XEP-0199: XMPP Ping - 3 user connections, each using a different node.

Test 22

Test 23

success

0 sec

3

81 ms

XEP-0030: XMPP Service Discovery - 3 user connections, each using a different node.

Test 23

Bosh 1

success

0 sec

3

4 ms

Socket connect to all cluster nodes.

Bosh 1

Bosh 2

success

0 sec

3

31 ms

Stream open to all cluster nodes.

Bosh 2

Bosh 3

success

0 sec

3

73 ms

SASL authorization to all cluster nodes to different accounts.

Bosh 3

Bosh 4

success

0 sec

3

74 ms

SASL authorization to all cluster nodes to the same account 'tester'.

Bosh 4

Bosh 5

success

0 sec

3

117 ms

Message send within one node to self user.

Bosh 5

Bosh 6

success

1 sec

3

236 ms

Message send within one node from 'admin' to 'tester' and wait for a response.

Bosh 6

Bosh 7

success

2 sec

3

468 ms

Message send within 'shell2.tigase.org' node from 'admin' and from 'all-xmpp-test_1'to 'tester' and wait for a response.

Bosh 7

Bosh 8

success

2 sec

3

417 ms

Message send within 'test-i.tigase.org' node from 'admin' and from 'all-xmpp-test_1'to 'tester' and wait for a response.

Bosh 8

Bosh 9

success

2 sec

3

443 ms

Message send beetwen 3 nodes: from 'shell2.tigase.org', user 'admin' and from 'test-i.tigase.org', user 'all-xmpp-test_1' to user 'tester' on 'veloci.tigase.org' and wait for a response.

Bosh 9

Bosh 10

success

1 sec

3

264 ms

Message send beetwen 3 nodes: from 'test-i.tigase.org', user 'admin' and from 'veloci.tigase.org', user 'all-xmpp-test_1' to user 'tester' on 'shell2.tigase.org' and wait for a response.

Bosh 10

Bosh 11

success

1 sec

3

289 ms

Message send beetwen 3 nodes: from 'veloci.tigase.org', user 'admin' and from 'shell2.tigase.org', user 'all-xmpp-test_1' to user 'tester' on 'test-i.tigase.org' and wait for a response.

Bosh 11

Bosh 12

success

0 sec

2

186 ms

2 user connections, each using a different resource on 'veloci.tigase.org' node.

Bosh 12

Bosh 13

success

0 sec

2

240 ms

2 user connections, each using a different resource on 'shell2.tigase.org' node.

Bosh 13

Bosh 14

success

0 sec

2

209 ms

2 user connections, each using a different resource on 'test-i.tigase.org' node.

Bosh 14

Bosh 15

success

1 sec

2

286 ms

2 user connections, on node 'veloci.tigase.org' and 'shell2.tigase.org'.

Bosh 15

Bosh 16

success

1 sec

2

294 ms

2 user connections, on node 'shell2.tigase.org' and 'test-i.tigase.org'.

Bosh 16

Bosh 17

success

1 sec

2

282 ms

2 user connections, on node 'veloci.tigase.org' and 'test-i.tigase.org'.

Bosh 17

Bosh 18

success

0 sec

3

43 ms

XEP-0092 get-version - 3 user connections, each using a different node.

Bosh 18

Bosh 19

success

0 sec

3

163 ms

XEP-0039 Statistics Gathering - 3 user connections, each using a different node.

Bosh 19

Bosh 20

success

0 sec

3

124 ms

Test for feature-not-implemented error - 3 user connections, each using a different node.

Bosh 20

Bosh 21

success

0 sec

3

136 ms

XEP-0199: XMPP Ping - 3 user connections, each using a different node.

Bosh 21

Bosh 22

success

0 sec

3

137 ms

XEP-0030: XMPP Service Discovery - 3 user connections, each using a different node.

Bosh 22

Prep 1

success

0 sec

1

0 ms

Setup a user daemon wating for messages

Prep 1

Multi 1

success

0 sec

30

23 ms

Multi-thread test of server Version info on node 'veloci.tigase.org'

Multi 1

Multi 2

success

3 sec

30

131 ms

Multi-thread test of server Version info on node 'shell2.tigase.org'

Multi 2

Multi 3

success

1 sec

30

42 ms

Multi-thread test of server Version info on node 'test-i.tigase.org'

Multi 3

Multi 4

success

0 sec

30

30 ms

Multi-thread test of server Version info on all nodes: 'veloci.tigase.org', 'shell2.tigase.org' and 'test-i.tigase.org'

Multi 4

Multi 5

success

2 sec

30

84 ms

Multi-thread test of server Statistics info on node 'veloci.tigase.org'

Multi 5

Multi 6

success

17 sec

30

576 ms

Multi-thread test of server Statistics info on node 'shell2.tigase.org'

Multi 6

Multi 7

success

7 sec

30

240 ms

Multi-thread test of server Statistics info on node 'test-i.tigase.org'

Multi 7

Multi 8

success

3 sec

30

128 ms

Multi-thread test of server Statistics info on all nodes: 'veloci.tigase.org', 'shell2.tigase.org' and 'test-i.tigase.org'

Multi 8

Multi 9

success

37 sec

90

415 ms

Multi-thread test of server Statistics info. Each user makes 3 connection, each on a different node and different resource: 'veloci.tigase.org', 'shell2.tigase.org' and 'test-i.tigase.org'

Multi 9

Multi 10

success

4 sec

30

163 ms

Multi-thread test Roster management functions on node 'veloci.tigase.org'

Multi 10

Multi 11

success

26 sec

30

886 ms

Multi-thread test Roster management functions on node 'shell2.tigase.org'

Multi 11

Multi 12

success

12 sec

30

412 ms

Multi-thread test Roster management functions on node 'test-i.tigase.org'

Multi 12

Multi 13

success

6 sec

30

228 ms

Multi-thread test Roster management functions on all nodes: 'veloci.tigase.org', 'shell2.tigase.org' and 'test-i.tigase.org'

Multi 13

Multi 14

success

9 sec

30

331 ms

Multi-thread test privacy lists management on node 'veloci.tigase.org'

Multi 14

Multi 15

success

41 sec

30

1391 ms

Multi-thread test privacy lists management on node 'shell2.tigase.org'

Multi 15

Multi 16

success

21 sec

30

709 ms

Multi-thread test privacy lists management on node 'test-i.tigase.org'

Multi 16

Multi 17

success

10 sec

30

334 ms

Multi-thread test privacy lists management on all nodes: 'veloci.tigase.org', 'shell2.tigase.org' and 'test-i.tigase.org'

Multi 17

Multi 18

success

2 sec

30

77 ms

Multi-thread test of message send to daemon user on node 'veloci.tigase.org'

Multi 18

Multi 19

success

12 sec

30

414 ms

Multi-thread test of message send to daemon user on node 'shell2.tigase.org'

Multi 19

Multi 20

success

29 sec

30

973 ms

Multi-thread test of message send to daemon user on node 'test-i.tigase.org'

Multi 20

Multi 21

success

4 sec

30

141 ms

Multi-thread test of message send to daemon user on all nodes: 'veloci.tigase.org', 'shell2.tigase.org' and 'test-i.tigase.org'

Multi 21

Multi 22

success

26 sec

90

274 ms

Multi-thread test of message send to daemon user. Each user makes 3 connections to all cluster nodes using different resources: 'veloci.tigase.org', 'shell2.tigase.org' and 'test-i.tigase.org'

Multi 22

Test end time: Oct 21, 2008 12:46:40 AM