Monitoring camel routes using camel-bam

Last week I got a question from an Apache Camel user about ways to get statistics on the activity on a route, including errors. There are quite a few ways to achieve that. First there are obviously the logs, but they are mostly useful to investigate a particular problem. Then there are the performance counters (defined in the org.apache.camel.api.management package) that could be obtained either via the JMX support or even exposed as a RESTful service. If one wants to be more fancy she could use a custom ErrorHandler.

There are however situations when message processing is faulty even when all individual messages are processed correctly. As an example, messages not processed in time can lead to service level agreement violations, or messages may be missing, or other business process execution use cases. While not pretending to compete with more specialized tools for business processes (such as BPEL or BPM engines) and while being inherently stateless, camel does offer support for scenarios like the ones described above, via the camel-bam component.

The BAM component is actually one of the rare exceptions that implements a stateful engine using JPA and a database of your choice. It also offers a very neat way to correlate messages by using Expressions (typical camel style) that evaluates against a message to produce a value used for correlation. As such, messages don't necessarily need to have an explicit correlation identifier as long as one can be computed from the content of the message, which means that it can normally be retrofitted to support systems not designed with correlation in mind. At its core camel-bam provides a set of temporal rules that trigger events. These events are processed by camel via processes (called activities) that are nothing else but specialized routes. BAM also offers a dsl for building activities via a ProcessBuilder (which is actually a specialization of the RouteBuilder you may be more familiar with).

That said, I put together a small example in my github account. It illustrates some very basic banking operations (like debits and credits from an account). The relevant part of the process is in ActivityMonitoringBuilder.configure() and looks something like this:

  ActivityBuilder teller = activity("direct:monitor-teller")
    .name("teller").correlate(xpath("/operation/@account"));
  ActivityBuilder office = activity("direct:monitor-office")
    .name("office").correlate(xpath("/operation/@account"));
  ActivityBuilder errors = activity("direct:monitor-errors")
    .name("errors").correlate(header("SampleRef"));
        
  office
    .starts().after(teller.completes())
    .expectWithin(Time.seconds(1))
    .errorIfOver(Time.seconds(1)).to("mock:overdue");
        
  errors
    .starts().after(teller.completes())
    .expectWithin(Time.seconds(1))
    .errorIfOver(Time.seconds(5)).to("mock:error");



The trigger events are generated when regular message flow down the routes defined in the BusinessProcessBuilder.

If you have more questions about how this example works (or bam in general) drop me a note or ask on the camel users@ lists. Enjoy!



Apache Rave a new ASF Top Level Project

After one year of hard work, Apache Rave graduated from incubation. Although the resolution was approved by the ASF board last week, the official announcement came only this morning (also featured on NASDAQ GlobeNewswire).

First initiated by Ate Douma and discussed at ApacheCon 2010 in Atlanta, the idea quickly captured the imagination of a group of developers and the proposal was submitted in February 2011. After a bit more then a year and a few releases, Rave is a promising top level project.

Dubbed a "web and social mashup engine", Rave is a powerful yet lightweight, secure, customizable platform for services and content delivery also supporting a number of social network technologies. Due to its content aggregation capabilities via specs like OpenSocial or W3C Widgets, Rave is already adopted by a number of portal projects. As content aggregation happens in the browser, many of the issues of traditional portlet based technologies are avoided.

For those those interested in a versatile platform for content delivery, check out Rave. Congrats to the Rave team for reaching this milestone and good luck going forward!

How to protect the release GPG key

Recently I have been asked about how I handle the gpg key I use for Apache releases. For what is worth, the question popped up in the context of a few other community members taking on the release manager role. As those of you that follow Apache Camel already know the Camel PMC decided to actively support and issue patch releases for the two latest minor branches.

But I digress. As I mentioned, I don't keep my private key on my laptop, but on an encrypted usb flash disk. The main reason is security, as the probability of someone getting access to my box greater than zero. In particular the key used for Apache releases is trusted by other ASF members and making sure it doesn't get compromised is one of the duties of the release manager. Of course one could revoke a key, but then verifying the integrity of a release becomes complicated at best.

My setup works on Ubuntu 11.10 and the idea behind it was using something similar to 2-factor authentication (something I have and something I know). I use a relatively cheap 4G usb flash memory (a sturdier, metal one). My usb flash uses a FAT32 file system and is not encrypted, but on it I created a 256M file (ringo.img) as an encrypted ext3 disk partition. My usb flash is mounted as '/media/APACHE', not much to do about the uppercase, a perk of FAT32. The encryption uses 256-bit aes and sha512. Below are the commands I used to create my encrypted partition (inspired from this article).

sudo apt-get install cryptsetup
cd /media/APACHE
export usbdisk=$PWD
dd if=/dev/zero of=$usbdisk/ringo.img bs=1M count=256

sudo modprobe cryptoloop
sudo modprobe dm-crypt
sudo modprobe aes_generic
export loopdev=$(sudo losetup -f)
sudo losetup $loopdev $usbdisk/ringo.img
sudo cryptsetup -c aes -s 256 -h sha512 -y create usbkey $loopdev

sudo mkfs.ext3 /dev/mapper/usbkey
sudo mkdir -p /media/encrypted
sudo mount -t ext3 /dev/mapper/usbkey /media/encrypted

After that, the encrypted media should be working an mounted as '/media/encrypted'. The article explains how to write mount.sh and umount.sh scripts to automate the mounting and unmounting of the encrypted media. You need to make sure you don't forget to unmount your encrypted partition to avoid data loss, although the risk is in good part mitigated by the use of a journaled file system.

Next is to setup gpg to use the keys from the encrypted file system. If you already use gpg, it's very simple. It stores the keys in a .gnupg directory in a user's home. The simplest way is to move the .gnupg directory on the encrypted media and create a softlink to it. As I want to be able to perform public key operations (encrypt/verify) even when my usb flash is not mounted, this is what I did:

mkdir ~/.gnupg-local
cp ~/.gnupg/pubring.gpg ~/.gnupg-local
cp ~/.gnupg/trustdb.gpg ~/.gnupg-local
sudo mv ~/.gnupg /media/encrypted
ln -sf /media/encrypted/.gnupg ~/.gnupg
# unmount ringo
sudo ln -sf ~/gnupg-local /media/encrypted/.gnupg

Note that the last command was executed after unmounting the encrypted partition. That way, if the encrypted partition is mounted the .gnupg softlink in my home will point to it, otherwise it will point to the softlink in the /media/encrypted which points back to ~/.gnupg-local.

What I did is something slightly different, I wanted to avoid issuing a manual command every time I plug the usb flash in. The system that manages dynamic devices is udev, so I had to write a udev rule. I made (renamed) copies of the mount and umount scripts in /usr/local/sbin and got rid of sudo in the scripts

-rwxr-xr-x  1 root root  449 2012-01-08 22:29 mount-ringo
-rwxr-xr-x  1 root root  167 2012-01-08 22:29 umount-ringo

I added a udev rule:

hadrian@rem:~$ cat /etc/udev/rules.d/100-mount-ringo.rules 
ACTION=="add", SUBSYSTEMS=="usb", ATTRS{idVendor}=="1307", ATTRS{idProduct}=="0165", RUN+="/usr/local/sbin/mount-ringo"

The idVendor and idProduct values you can get from `lsusb`. They make sure the rule only fires when you mount the right usb flash memory. You can use other attributes to identify your flash if you prefer, like the label for instance.

hadrian@rem:~$ lsusb
[...] 
Bus 003 Device 018: ID 1307:0165 Transcend Information, Inc. 2GB/4GB Flash Drive

You can test your rules using `udevadm`. For that you need to know the mounted device which you can get if you know the label ('sde' in my case). If everything is ok, you should see a 'run: <your-script>' message near the end.

hadrian@rem:~$ ls -al /dev/disk/by-label/APACHE 
lrwxrwxrwx 1 root root 9 2012-01-09 11:13 /dev/disk/by-label/APACHE -> ../../sde
hadrian@rem:~$ sudo udevadm test /sys/block/sde
run_command: calling: test
[...]
ID_VENDOR_ID=1307
ID_MODEL_ENC=USB2FlashStorage
ID_MODEL_ID=0165
ID_FS_LABEL=APACHE
ID_FS_LABEL_ENC=APACHE
ID_FS_UUID=6F8A-7C90
ID_FS_UUID_ENC=6F8A-7C90
ID_FS_VERSION=FAT32
ID_FS_TYPE=vfat
ID_FS_USAGE=filesystem
[...]
run: '/usr/local/sbin/mount-ringo'
run: '/lib/udev/hdparm'
run: 'socket:@/org/freedesktop/hal/udev_event'

Once you are happy with your test you can restart udev to use your new rule:

hadrian@rem ~$ sudo udevadm control --reload-rules

As a backup, you may want to put your private key on a non-encrypted usb flash that you don't use and is kept in a secure location. Now you should be all set, just pull your usb flash out and put it in your pocket and remember to always umount first. Enjoy!