Exchange View: Blocked Exchanges very difficult (impossible?) to test

[phantomjinx]

The Exchanges view displays the statistics of both Inflight and Blocked Exchanges as 2 distinct tables. The Inflight Exchanges are straightforward to test by adding a Processor to a route that calls Thread.sleep(60000). The Exchanges that are inflight but pending due to the sleeping Processor are logged as rows in the Inflight Exchanges table.

However, despite best attempts (see here), trying to create a test app that creates blocking threads that are detectable and so listed in the Blocked Exchanges table is very complex (near impossible??).

The architecture for extracting the Blocked Exchanges is to interrogate the MBean -> Service -> DefaultAsyncProcessorAwaitManager. This mbean contains the count of the number of threads blocked as well as a list of the exchanges. However, in common routes, this particular Manager seems to be rarely implemented (breakpoints on it never get called) and its key method that seems to log the blocked threads seems never to be called.

(It should be noted that it is possible to override this class with a custom AsyncProcessorAwaitManager which means that a different name is listed in MBean -> Service and so breaks the logic of extracting the bean by name.)

Looking at the test example it is not difficult to block threads and so exchanges. However, these blocked threads are never logged by this mbean and so never displayed in the table. Therefore, perhaps a rethink of this specific table may be required to make it more useful?

[tadayosi]

@jsolovjo Do you know if this feature is covered in the existing test suite?

@phantomjinx I think it's great to cover the feature in the test suite. On the other hand, considering it just invokes a backend MBean and render the results in almost the same way with the Inflight Exchanges, for the time being we can just rely on DefaultAsyncProcessorAwaitManager backend MBean and implement the table in the same way and move on. We can revisit the issue later in the QA phase.

[tadayosi]

(It should be noted that it is possible to override this class with a custom AsyncProcessorAwaitManager which means that a different name is listed in MBean -> Service and so breaks the logic of extracting the bean by name.)

Actually, the legacy impl looks up the MBean by the wildcard DefaultAsyncProcessorAwaitManager*, so possibly if we provide a custom impl with a dummy name like org.apache.camel:type=services,name=DefaultAsyncProcessorAwaitManagerForHawtioTest can we test the feature?

And btw, have you enabled statistics for the test Camel route? It looks like this flag is required for collecting blocked exchanges.
https://camel.apache.org/manual/jmx.html#_configuring_performance_statistics

[phantomjinx]

So if you implement a new Class that implements AsyncProcessorAwaitManager then call context.setAsyncProcessorAwaitManager(manager) then the name can be completely custom, eg. MyAsyncProcessorAM. It is this name that becomes the name of the mbean. The need for this to happen though is perhaps so rare that our concern should be very low priority for the moment.

I did enable statistics on everything I came across. That being said, the doc page references context.getManagementStrategy().setEnab... and yet that particular interface nor its implementing classes Default / JMX contain that method.

Either way my attempt should provide a starting point for further testing.

[jsolovjo]

Hi @tadayosi
Our E2E tests cover basic scenarios including Exchanges but missing complex configurations - especially, this one.
To be honest, we thought that No blocked exchanges is expected and didn't pay big attention to it.

If you could help me to understand it and explain how it works, and what expected configurations and user behaviours are - it would be very beneficial and will help me to create more meaningful tests to cover complex scenarios. (Not only related to this specific issue)

Thank you

[tadayosi]

@jsolovjo Thanks. While it is our consensus that it's hard to replicate blocked exchanges, inflight exchanges in the Exchanges view is a good target to cover in the E2E tests at hawtio/hawtio. Paul already provides a nice reproducer for the inflight exchanges here, so you can start from this and add some test scenario in the hawtio/hawtio E2E tests:
https://github.com/phantomjinx/hawtio/tree/main/examples/springboot-pgr/src/main/java/io/hawt/example/spring/boot
This should work for both Hawtio legacy and NG. If you need more detail about this feature, can you please talk with Paul?

[jsolovjo]

@tadayosi Thank you. I will have a look at this and implement tests to cover these scenarios.
Right now we are busy with the builds and test result evaluations, so I will come back to it next week.