Tracing applications with the IBM MQ Appliance - Middleware News

The MQ Appliance adds a new simpler way to start collecting this trace information. It also gives you new ways to narrow down what you actually trace.

Application activity trace has been available on the distributed MQ platforms since version 7.1 and is a mechanism that allows you to track exactly what an application is doing when it comes to its MQ operations. So it'll show you how they connect, what queues or topics they open and then the messages they put and get. An excellent way to diagnose a wide variety of problems.

If you're already familiar with activity trace you'll know that you can enable it in a number of ways and once it's collecting trace it'll write all the trace data as a series of messages to the SYSTEM.ADMIN.TRACE.ACTIVITY.QUEUE of the queue manager. There are multiple ways to enable activity trace, it's either through turning trace on for all applications by using the queue manager attribute ACTVTRC, or through writing rules in the queue manager's mqat.ini file or even by programmatically enabling it with a connect option. The most useful is probably the rules in mqat.ini as they allow you to scope trace down to certain named applications in a pretty dynamic way but it does require direct access to the machine to modify the mqat.ini file, which is not something that you can do with the MQ Appliance. So as I said, instead of writing rules in mqat.ini there's a new way to enable trace, and this is even more dynamic and more versatile in scoping what is traced than the mqat.ini rules. It also overcomes the limitation of having all trace data go to that single SYSTEM.ADMIN.TRACE.ACTIVITY.QUEUE that the existing ways have.

The MQ Appliance supports a new set of system topic strings. These all start "$SYS/MQ". And one of the reasons for these is that when you subscribe to topic strings under a special branch of those topics you will start to receive application activity trace for the thing you've just subscribed to. And as with all subscriptions, the messages will be sent direct to your subscription's queue. And then when you delete the subscription the tracing will automatically stop. And none of this requires queue manager or application changes at all to turn it on and off, the existence (or not) of the subscriptions to these special topic strings is enough. The other benefit is that you can have multiple subscriptions to the same applications and they each receive their own copy of the data.

The topic strings for activity trace all start with the following:

$SYS/MQ/INFO/QMGR/gr name>/ActivityTrace

(Where is obviously the name of your queue manager)

I also mentioned news scopes of tracing, not just be application name. So you then have three options of what to match against, you either trace based on an application's name, a channel name or an MQ connection's unique Id. So for example...

$SYS/MQ/INFO/QMGR/mgr name>/ActivityTrace/ApplName/amqsputc.exe
$SYS/MQ/INFO/QMGR/mgr name>/ActivityTrace/ChannelName/SYSTEM.DEF.SVRCONN
$SYS/MQ/INFO/QMGR/mgr name>/ActivityTrace/ConnectionId/414D5143514D475231202020202020206B576B5420000701

The ApplName method is the same as you have with the existing mqat.ini rules on the other platforms, so you simply add the trailing part (following the last slash) of what you'd see in the APPLTAG of the connection, or the PutApplName of the MQMD.

The ChannelName is new to activity trace, it means you can use it to trace everything going on or coming off a sender or receiver channel between two queue managers. You can also use it to see what client applications are doing over a particular SVRCONN channel, in which case you'll see all the MQI calls, not just the messages flowing.

And finally there's ConnectionId. This one allows you to target a specific MQ connection. So if you've identified an existing connection to a queue manager that you want to investigate further you can provide its unique Id and you'll start to receive details of what's going on on that connection. In case you're wondering, you build that unique Id by concatenating the EXTCONN and CONN values returned from DISPLAY CONN in runmqsc.

As we're simply using the existence of the subscription to start and stop generating the trace data then you obviously have various options for how you use this, for example you could administratively define a subscription to gather the trace for a prolonged period of time, and you post-process the data generated and sent to the subscription queue. Alternatively you could use the sample we provide in the 8.0.0.2 client download, amqsactc, to dynamically enable trace for the duration that it is running. It does this through creating a non-durable subscription, which enables the trace, formatting the published data that arrives on the subscription queue direct to the screen, and turning trace back off again when the application stops by it simply deleting the subscription (as it's non-durable killing the sample has the same effect).

To make that easy we've extended the sample to allow you to say what you want to trace, by application name (-a), channel name (-c) or connection Id (-i), without needing to remember those long topic strings....

amqsactc -m QMGR1 -w 30 -a amqsputc.exe
amqsactc -m QMGR1 -w 60 -c SYSTEM.DEF.SVRCONN
amqsactc -m QMGR1 -w 600 -i 414D5143514D475231202020202020206B576B5420000701

(And for the keen eyed, the "-w" tells amqsactc to run for a number of seconds and then stop which is necessary when running in this 'live' trace mode as you'll need to be collecting trace for a period of time.)

You'll find more documentation on application activity trace here and information of these new special system topic strings here. So if you get your hands on an appliance, take a look at this feature.