apifocal offers professional services focused on scalability, security and maintainability all based on open source technology

, ,

A JMS Ping Utility

Most companies that do integration at scale use some variation of an asynchronous messaging service. JMS is by far the preferred choice, whether using commercial products like IBM WebSphere or Oracle WebLogic (to name the more popular commercial ones, there are a few others), or open source alternatives, of which Apache ActiveMQ is by far the most popular. So popular, in fact, that Amazon included it a few months ago in its cloud services menu.

Regardless if one operates a messaging service on premise or in the cloud, it is important to constantly monitor and assess the health of the service. Unfortunately, tools that support operations of messaging services are virtually nonexistent, or have a limited scope at best. Therefore, every organization comes with its own solution. Over the years, we developed a nifty tool set that we are moving to open source under the business friendly Apache License v2.

A simple and very useful tool is a JMS ping utility. It is intended for ActiveMQ, but the code only uses the javax.jms package. With minor changes (mostly dependencies) it can be adapted for use with other messaging products.

Assuming you have a broker running, you can execute something like:

The tcp://localhost:60001 address is the connection URL for the broker. If you don’t have a broker running, you can easily run a local broker from the test-brokers directory in the project by running:

The jms-broker profile from the maven project runs a local minimal broker.

The JMS ping utility is very useful in measuring the health of a broker. One may run it occasionally in development, but I find it extremely useful for production operations to measure, more or less continuously, the response time of one or multiple brokers. The amix-ping tool help prints:

The tool works with both authenticated and anonymous connections. For authenticated connections the -u and -p parameters are mandatory. Although the majority of brokers I’ve seen in production environments use, surprisingly, anonymous connections, I strongly recommend using authenticated connections in production. Yes, credential management may be a pain, but the added security is well worth it.

The -c argument allows for multiple pings being sent. The default is 1, as seen in the sample output above, but in general we want to send more pings and measure the average time.

The -i argument throttles down the pings to an interval specified, in seconds. By default, the messages are still throttled down to 20 ms (to a max of 50 msg/sec) to avoid flooding the broker.

The -l,--length is an interesting one. Predictably, it allows one to send random payloads of the specified size (in bytes), the default being 100. What is interesting is that this allows one to measure the response of the broker for various size loads. As an example (4 msg bursts):

What we see is that the first message takes longer – that’s not surprising given how ActiveMQ internals work. The 1 ms time is because the broker runs on the same host with no persistence. One thing to notice is that there is no significant difference between 100 bytes and 1 kb payloads, but once the payload size increases, so does the ping round trip time. This is not a surprise either, but what is important to know is by how much to fine tune the broker performance.

Running the JMS ping utility against a production broker, like SilkMQ, gives more realistic results. SilkMQ is apifocal’s managed ActiveMQ network of brokers service that currently runs distributed among multiple datacenters on 2 continents, including AWS.

The ping time for a broker on the US east coast is around 15 ms for small payloads, but this utility allows me to measure for different payload size and, more importantly, broker saturation. For that I run the ping from multiple consoles on my laptop, or I can run it from computers in different geographies. If I ping from two consoles in parallel, I will probably not see a degradation in response time vs just running one ping instance. Once I increase the number of parallel ping sessions, at some point we’ll start noticing a slight performance degradation, and even a serious degradation as the number of ping sessions grows. This is very helpful for tuning the performance of the brokers, in this case by configuring the ActiveMQ producer flow control.

One thing you may have noticed in the output is the destination name. From the usage text above, we see that the destination name may be provided as an optional parameter. The default is queue://jms.ping but we see that the actual destination has a random 6 character string appended to the name. The reason is that the ping utility uses both a producer and a consumer, and time is measured by using the JMSTimestamp property, so we want to make sure we don’t receive the timestamp from another computer running a ping at the same time (the time difference would be meaningless, right?).

Since we mentioned JMS message properties, there is one more thing worth mentioning. The ping utility also sets the TTL of the message to 2 min, so there is no need to flush destinations in case something went wrong. The broker will discard pings after 2 mins automatically, freeing up resources.

At the time of writing, the first version of activemq-mix open source utilities is not yet released (although we used the code in production for a long time). If you find this post after the release, the README will provide more up-to-date instructions. Otherwise, some scripts may be missing, including the amix-ping one seen above. You can get by by dropping a shell script with the content below on the $PATH.

Enjoy, and if you find this utility helpful, drop us a note. Comments and constructive feedback encouraged.

,

Hello apifocal

apifocal delivers professional services and products for developing and operating infrastructure services at scale

It is impossible to browse news or articles in the IT realm without hitting the term ‘API’ at every corner. And in spite of this, the mention of APIs doesn’t leave readers with that annoying aftertaste of marketing hype. The reason is that it’s hard to overemphasize APIs as the enabler of integration – the secret ingredient for creating higher order, higher value services.

With that principle in mind we started apifocal a couple of years ago. The name is a pun on epifocal, defined by dictionary.com as “situated or occurring at an epicentre” and is pronounced the same way.

apifocal (yes, the lower case is intentional) delivers professional services and products for developing and operating infrastructure services at scale, particularly in the healthcare and finance sectors. In our experience, many organizations struggle to bridge the gap from proof-of-concepts or small scale operations to large scale. And that is not surprising, because the economics of operating at scale are very different and require a level of infrastructure that is usually ignored at no scale.

Future posts will delve into scalability aspects – we have a lot to share. For now, apifocal just wants to say “Hello World”.