Project

General

Profile

filter development using AnnotatedXMPPProcessor and XMPPPacketFilterIfc

Gagan Malik
Added about 2 years ago

Hi,

I'm trying to develop a very basic packet filter to intercept all xmpp stanzas. Let's call this filter_tako. For now, it just logs the packet in the filter() method. This is loaded in the init.properties with

--sm-plugins=+message,+message-carbons,+filter_tako,+other_custom_plugin

Here are the questions I have

1.) Despite the @Handles and @Handle annotation from the eg (http://docs.tigase.org/tigase-server/7.1.0/Development_Guide/html_chunk/pluginDev.html#_using_annotation_support) the filter intercepts 'iq', 'presence' and 'message' stanzas. I even tried overriding supElements() method to return "messages", but the filter still intercepts non-messsage stanzas and prints them. What am I missing?

2.) I then tried to overcome this in an overridden canHandle() method with the following check, which didn't work either.

StringUtils.equals(tigase.server.Message.ELEM_NAME, packet.getElemName())

3.) A yet another unexpected thing I encountered was that the filter_tako was invoked multiple times from many different threads, for instance [session-open Queue Worker 6], [in_43-sess-man], [message-carbons Queue Worker 0], [presence-state Queue Worker 0], [in_16-sess-man], [other_custom_plugin Queue Worker 0], [message Queue Worker 0]. I say this because I saw the log statement in the filter_tako pre-pended with these worker names. for instance,

2017-02-22 17:23:03,077 [presence-state Queue Worker 15] DEBUG filter_logger - filter_tako |
packet: from=ws2s@{host}/172.31.0.63_5290_172.31.0.1_41443, to=sess-man@{host},
DATA=<presence xmlns="jabber:client" from="{jid}/F3076E56-985B-4610-89B4-9926D42C2C37" type="unavailable"/>,
SIZE=145, XMLNS=jabber:client, PRIORITY=PRESENCE, PERMISSION=AUTH, TYPE=unavailable

Shouldn't this be processed only twice as mentioned at http://docs.tigase.org/tigase-server/7.1.0/Development_Guide/html_chunk/packetProcess.html


Here's the code I have..

@Id(TakoFilter.FILTER_ID)
@Handles({
    @Handle(path = {tigase.server.Message.ELEM_NAME}, xmlns = XMPPProcessorIfc.CLIENT_XMLNS)
})
public class FilterTako extends AnnotatedXMPPProcessor implements XMPPPacketFilterIfc {

    protected static final String FILTER_ID = "filter_tako";

    @Override
    public Authorization canHandle(Packet packet, XMPPResourceConnection conn) {

        Authorization result = super.canHandle(packet, conn);

        if (result == Authorization.AUTHORIZED) {
            if(!StringUtils.equals(tigase.server.Message.ELEM_NAME, packet.getElemName())) {
                return null;
            }
        }
        return result;
    }

    @Override
    public void filter(Packet packet, XMPPResourceConnection session, NonAuthUserRepository repo,
            Queue<Packet> results) {
        logger.debug("{} | packet: {}", FILTER_ID, packet.toString());
    }
}

Replies (5)

Added by Andrzej Wójcik IoT 1 CloudTigaseTeam about 2 years ago

All questions you posted are related to fact that you are using XMPPPacketFilterIfc which intercepts every packet and results of processing done by @XMPPProcessor@s.

1) and 2) As I mentioned above, you get every packet, so you it is up to you select packets you are interested in within filter() method. Method canHandle() and annotations (which in fact are used within implementation of canHandle() method are used for filtering only packets delivered to @XMPPProcessorIfc@.

3) As I mentioned above: "@XMPPPacketFilterIfc@ intercepts every packet and results of processing done by XMPPProcessor@s", so if more than one processor is processing packet then @filter() will be called after each processor finishes it's task (as same packet is processed in same time by multiple processors).

As for your goal:

I'm trying to develop a very basic packet filter to intercept all xmpp stanzas

I think it could be better if you would use XMPPPreprocessorIfc as it is called only once for single processing of a <message/> by SessionManager, but it is done before packet is processed by SM. Also preprocessors need to be very fast.

As for @ XMPPPacketFilterIfc@ you can get similar results but you would need to deal with multiple executions of filter() method for same packet being processed, but you also get results of processing packet by SM.

Added by Gagan Malik about 2 years ago

Thank you Andrzej! That explains a lot. I have these follow up questions now..

1.) What is then a real world use-case of XMPPPacketFilterIfc@? I've seen it implemented in a few classes in the @tigase.xmpp.impl but that is almost always along with XMPPProcessorIfc and XMPPPreprocessorIfc except in ErrorCounter@. Wouldn't ErrorCounter@ErrorCounter be susceptible to counting the same error packet multiple times?

2.) The end goal is to develop a spam-block functionality. Now, pre-processor does seem the right place after your explanation, but there will need to be at least one synchronous http request made to a 'spam-check service' to determine if the message should be let through or be blocked. Do you still think pre-processor is the right place?

3.) This is related to question #2. I referred to an earlier discussion on the forums here https://projects.tigase.org/boards/4/topics/3422

Like you have pointed out that pre-processor needs to be very fast, Artur mentions

You can do the same thing in the "filer" as you do in the "preprocessor". You can amend the message or block it completely form delivering. The main difference is that you do it later in the processing chain. This may mean like a bad idea as if you block something you want to do it as quickly as possible. However, the main advantage is that "filter" is being processed in the plugin thread pool. Therefore it does not impact main processing in SessionManager, therefore it overall impact on the server performance is much lower.

What is the best approach, if any, to de-duplicate the multiple executions of @filter()@, if that is the correct place for such functionality?

Thanks again for your help.

Added by Andrzej Wójcik IoT 1 CloudTigaseTeam about 2 years ago

Gagan Malik wrote:

Thank you Andrzej! That explains a lot. I have these follow up questions now..

1.) What is then a real world use-case of XMPPPacketFilterIfc@? I've seen it implemented in a few classes in the @tigase.xmpp.impl but that is almost always along with XMPPProcessorIfc and XMPPPreprocessorIfc except in ErrorCounter@. Wouldn't ErrorCounter@ErrorCounter be susceptible to counting the same error packet multiple times?

No, as it processes results packets (packets sent by SM).

2.) The end goal is to develop a spam-block functionality. Now, pre-processor does seem the right place after your explanation, but there will need to be at least one synchronous http request made to a 'spam-check service' to determine if the message should be let through or be blocked. Do you still think pre-processor is the right place?

pre-processor is a right place but due to slow processing (synchronous call to http) it will not work as it should and may cause additional delays during processing of messages.

3.) This is related to question #2. I referred to an earlier discussion on the forums here https://projects.tigase.org/boards/4/topics/3422

Like you have pointed out that pre-processor needs to be very fast, Artur mentions

You can do the same thing in the "filer" as you do in the "preprocessor". You can amend the message or block it completely form delivering. The main difference is that you do it later in the processing chain. This may mean like a bad idea as if you block something you want to do it as quickly as possible. However, the main advantage is that "filter" is being processed in the plugin thread pool. Therefore it does not impact main processing in SessionManager, therefore it overall impact on the server performance is much lower.

What is the best approach, if any, to de-duplicate the multiple executions of @filter()@, if that is the correct place for such functionality?

I think it would be best to go with totally different approach and create preprocessor which forwards <message/> to different component which would be handling message filtering based on remote calls to HTTP server and then sending packets back to SM.

This solution will perform quite well in terms of performance but it would be tricky to implement it.

Thanks again for your help.

Added by Gagan Malik about 2 years ago

Thank you very much! I'll look into how to make that happen with your suggestion.

I have one more, possibly the last :), question here. Other than the fact that XMPPPacketFilterIfc works explicitly in the session manager, what are the differences between creating a filter using XMPPPacketFilterIfc v/s using PacketFilterIfc@? I guess what I'm asking is, like the pre-processor that works in the session manager thread (not a thread pool), does a filter created using @PacketFilterIfc work in the component's thread that it is loaded into? Does it need to be fast and possibly not do blocking http calls?

Thanks for clarifying so many things!

Added by Andrzej Wójcik IoT 1 CloudTigaseTeam about 2 years ago

I guess what I'm asking is, like the pre-processor that works in the session manager thread (not a thread pool), does a filter created using PacketFilterIfc work in the component's thread that it is loaded into?

Yes, they work in very similar way.

Does it need to be fast and possibly not do blocking http calls?

As it works in component thread pool it needs to be fast as well.

    (1-5/5)