Saturday, March 31, 2012

Monitoring camel routes using camel-bam

Last week I got a question from an Apache Camel user about ways to get statistics on the activity on a route, including errors. There are quite a few ways to achieve that. First there are obviously the logs, but they are mostly useful to investigate a particular problem. Then there are the performance counters (defined in the org.apache.camel.api.management package) that could be obtained either via the JMX support or even exposed as a RESTful service. If one wants to be more fancy she could use a custom ErrorHandler.

There are however situations when message processing is faulty even when all individual messages are processed correctly. As an example, messages not processed in time can lead to service level agreement violations, or messages may be missing, or other business process execution use cases. While not pretending to compete with more specialized tools for business processes (such as BPEL or BPM engines) and while being inherently stateless, camel does offer support for scenarios like the ones described above, via the camel-bam component.

The BAM component is actually one of the rare exceptions that implements a stateful engine using JPA and a database of your choice. It also offers a very neat way to correlate messages by using Expressions (typical camel style) that evaluates against a message to produce a value used for correlation. As such, messages don't necessarily need to have an explicit correlation identifier as long as one can be computed from the content of the message, which means that it can normally be retrofitted to support systems not designed with correlation in mind. At its core camel-bam provides a set of temporal rules that trigger events. These events are processed by camel via processes (called activities) that are nothing else but specialized routes. BAM also offers a dsl for building activities via a ProcessBuilder (which is actually a specialization of the RouteBuilder you may be more familiar with).

That said, I put together a small example in my github account. It illustrates some very basic banking operations (like debits and credits from an account). The relevant part of the process is in ActivityMonitoringBuilder.configure() and looks something like this:

  ActivityBuilder teller = activity("direct:monitor-teller")
    .name("teller").correlate(xpath("/operation/@account"));
  ActivityBuilder office = activity("direct:monitor-office")
    .name("office").correlate(xpath("/operation/@account"));
  ActivityBuilder errors = activity("direct:monitor-errors")
    .name("errors").correlate(header("SampleRef"));
        
  office
    .starts().after(teller.completes())
    .expectWithin(Time.seconds(1))
    .errorIfOver(Time.seconds(1)).to("mock:overdue");
        
  errors
    .starts().after(teller.completes())
    .expectWithin(Time.seconds(1))
    .errorIfOver(Time.seconds(5)).to("mock:error");


The trigger events are generated when regular message flow down the routes defined in the BusinessProcessBuilder.

If you have more questions about how this example works (or bam in general) drop me a note or ask on the camel users@ lists. Enjoy!

2 comments:

  1. Hi, i'm in process of adopting Camel BAM to our needs and found a curios thing - processed messages are not kept and there is no way to reference them in error Route. Maybe i'm missing something? Also what is the difference between expectedWithin and errorIfOver?

    Thanks

    ReplyDelete
  2. Here is my blog post about monitoring alternatives for Apache Camel:

    http://www.kai-waehner.de/blog/2013/07/15/apache-camel-and-talend-esb-management-and-monitoring-of-integration-routes-and-soap-rest-web-services-jmx-osgi-logstash-elasticsearch-kibana-hawtio

    My favorite is Kibana, a great open source project for creating dashboards based on log files.

    ReplyDelete