Project

General

Profile

Is JAXMPP client(in Tigase XMPP Client Library) suitable to do load performance test?

Rick Zhang
Added over 4 years ago

Hi:

Recently we are testing our tigase application.As we are not fimiliar with TSung,and we found there are a tool including jaxmpp-j2se-2.0.2 、jaxmpp-core-2.0.2.jar.The api is simple,so we write below code to test "login/logout" to our tigase application:

@Jaxmpp conn =new Jaxmpp();

conn.login(false);

...

while(!conn.isConnected()){

Thread.sleep(100);

}

conn.disconnect();@

we test case is:10 domain and 10 user in each domain.All of the 100 users login at the same time(executed by 100 threads).but the result is strange:some users always can't finish the job even we waiting for 10 minutes.The broken thread is loop in this section:

@while(!conn.isConnected()){

Thread.sleep(100);

}@

My problem is:

Is JAXMPP client suitable to do load performance test?I think it may be a SDK which is not suitable for creating/connecting many Jaxmpp Object in one machine.Am I right?Thank you for your answer.


Replies (6)

Added by Steffen Larsen over 4 years ago

It definitly depends of your load test and your expected users/sec and the load you want to push to your XMPP server.

But as I see the jaxmpp it is not that suitable for load testing, because you have to control the threads and setup monitoring etc. all this you get for free when using tsung.

Its actually not that hard to setup, just a simple xml file to describe your scenarios.

-Cheers

/Steffen

Added by Rick Zhang over 4 years ago

Thank you very much.I know that if use jaxmpp to test performance we have to do a lot of work.I will change to TSung soon.But this time I want to know if jaxmpp library exists problems in multi-thread environment(even in 100 thread).If the library is strong and stable in multi-thread and has been tested sufficiently,the test result in this topic illustrates my tigase application has some problem.But if the library is not stable enough,I need stop analysing my tigase application and change to TSung immediately.

Avatar?id=6023&size=32x32

Added by Artur Hefczyc TigaseTeam over 4 years ago

Yes, we do use JaXMPP for load testing sometimes. It allows us to create very custom and powerful test scenarios which are not possible with Tsung. Also Tsung is not really a Jabber client, it has no understanding of XMPP protocol and does not check correctness of what was received, etc... If you use JaXMPP you also have a better verification of the workflow and data correctness.

This comes at expense of higher resources requirements of course and more complex deployment and running such load tests. However, I was able to generate thousands of online users (not sure whether the limit was 10k or 20k) per machine. Which is quite good.

As for the monitoring, I do not care about Tsung ability of monitoring as they are not suitable for our needs. I can recommend to use Tigase Monitor which shows you load on the server in real time, users distribution, etc... Very useful for load tests. As for the deployment and running test from a multiple machines - a bunch of command line scripts work very well for us.

That's said, I admit, we use Tsung for majority of load tests.

Added by Matthew M over 4 years ago

Actually we are trying to user jaxmpp for load testing too, as you said, jaxmpp allows verification of XMPP with details.

I wonder if you share with us a some sample scripts or load test program, that we can quickly configure a number users and run jaxmpp client on a machine?

We have some problems when running multiple jaxmpp client threads, sometime a client login thread stuck there for a long time, and we found server logs stops too. It seems nothing is going on between that jaxmpp client and server. Is there any effective way to look into this problem, and find out whether it is the client set up issue, or the server issue?

Thanks!

Avatar?id=6023&size=32x32

Added by Artur Hefczyc TigaseTeam over 4 years ago

Matthew M wrote:

Actually we are trying to user jaxmpp for load testing too, as you said, jaxmpp allows verification of XMPP with details.

There are 2 kinds of tests (actually there are more but 2 I want to mention here):

  1. Functional (for strict workflow verification) tests for which we use our Tigase Test Suite, it is an open source project which allows you to design and implement own test scenarios and test cases. We run release builds through TTS and nightly builds: http://build.tigase.org/~tigase/ and: http://build.tigase.org/nightlies/tests/, the code is open source and there is some documentation for it.

  2. Load tests for which you are more interested in how the system behaves under a high load and less about correctness of the data processing: http://graph.cluster-c.xmpp-test.net/latest/ Here we use Tsung.

I highly recommend not to mix these 2 into a single test. In my opinion this is impossible task, or at least insanely expensive in terms of human and hardware resources.

I wonder if you share with us a some sample scripts or load test program, that we can quickly configure a number users and run jaxmpp client on a machine?

Unfortunately our JaXMPP code for load tests was only for a very custom use-cases, not easily reusable for a general load tests. For general load tests we use Tsung and I actually recommend using Tsung unless you have some specific needs not covered by Tsung.

We have some problems when running multiple jaxmpp client threads, sometime a client login thread stuck there for a long time, and we found server logs stops too. It seems nothing is going on between that jaxmpp client and server. Is there any effective way to look into this problem, and find out whether it is the client set up issue, or the server issue?

I would suggest to investigate which side is waiting for a response from the other side and why. Let's say client sent roster-get request and is waiting for a response - then check whether the server got the request and why it does not send response.... If you active debug logs on the server you get very good picture of all the data flows for each user. You can also activate debug mode on the client side to see what is going on.

Added by Rick Zhang over 4 years ago

Thanks all of you.I am clear now.Jaxmpp can be used in multi-thread environment and do some load performance test(at least it can support thousands of users in one machine as Artur Hefczyc did before.Now clearly there are some problems in my tigase server application.Thand you again.

    (1-6/6)