Code: Click here for accompanying code
This post assumes you know the basics – how to install MSMQ, create queues, setup appropriate security measures, and how to enqueue and dequeue messages using Microsoft .NET [via the System.Messaging namespace APIs]. If you haven’t used .NET APIs against MSMQ, the accompanying C# code should help jumpstart your efforts.
If you’re considering using MSMQ or are looking to improve the performance of an existing implementation, here are some notes from the field that can help speed up messaging:
Queue Data Location
Out of the box, MSMQ is setup to store queue data on the %system% drive. You should change this to a non-system drive; preferably to a different physical hard-drive. This may seem obvious, but is important enough for a mention. Apart from being a best practice, if left alone, your systems engineers would probably frown upon having a LOB application read/write from/to the system drive. Note that transactional queues will see a huge benefit by doing just this. There are other reasons you want to do this:
- Avoid the system drive filling up – If your application that dequeues messages is down, depending on the throughput of messages, there is a possibility of filling up your system drive. This could cause the server to slow down drastically or even make it unresponsive.
- SSDs: The industry trend is leaning towards having high-performance %system% disks such as Solid State Drives. You typically want to avoid excessive writes to a SSD.
- Cloud: System drives are usually not very big (for performance and/or cost reasons) – and this pattern seems to extend to the cloud as well. Hosting your queues on non %system% drives helps this setup.
All MSMQ messages are serialized to disk onto memory mapped files. Depending on the resiliency settings and the APIs used, writes to a queue can be synchronous or asynchronous. Here’s a MSDN blog article that explains this.
Since the disk plays an integral role in MSMQ, you have to do some groundwork to determine what kind of hardware you require to meet your application’s SLA. This accompanying code has a ‘MsmqTests’ project that can be used in your development, QA, and Production boxes to benchmark throughput. Make sure that the payload of the message-body, queue type, properties, and formatter attributes are aligned to your application needs.
Performance monitoring is your friend. Use PerfMon to look at resource utilization such as % Processor Time and Avg. Disk Queue length. It is noted that the average disk queue length higher than 0.6 may mean that additional physical disks may be needed. On the other hand, if your processor is consistently high, other resource optimizations and/or upgrades may be required.
Queue Types and Properties
Know what type of queue to use. If performance is pressing, a Private Queue without Transactions and the ‘ExactlyOnce’ property set to off is recommended. Determine if enabling transactions on the queue is necessary. Conducting transactional operations before enqueuing a message should be considered. This allows you to turn off the transactional attribute on the queue. Another reason I’ve encountered clients turning on transactions is EOD (exactly-once-delivery) for multi-threaded applications. Transactional queues inherently force EOD and ordered delivery.
When instantiating access to the queue in code, use the ‘Direct’ name format. This type of instantiation involves the least number of roundtrips to Active Directory and speeding up that process.
Formatters and Serialization
As a rule of thumb, I prefer using the ActiveX or Binary message formatter and serialize data to a primitive data-type. I found this technique to deliver the best performance. The XmlMessageFormatter provides for loosely coupled messaging; but, I found that a good alternative is using JSON or BSON over ActiveX or Binary formatters. Although I recommend keeping your messages short, there are scenarios where that’s not always possible – making serialization decisions complex. That’s when you need to conduct further tests to determine the best fit – Object serialization, JSON, or XML depending on how descriptive or loosely coupled you want the data to be.
Note that you have to use a new instance of a formatter for each read/write operation to/from the queue. Formatters are not thread safe and reuse can cause corrupted messages when you retrieve data out of the Body property (in .NET).
If you’re serializing data for clients on diverse technology stacks, Object serialization may be a bit more trickier to implement. As of this writing, I’ve encountered memory leaks and a whole bunch of other issues using J-Integra (for instance) to dequeue serialized object graphs and reconstruct them on the Java side. I found native types and/or JSON to be most performant in these scenarios.
To summarize, Disk, Queue Properties, Formatting, and Serialization are the primary factors I’ve found contributing to latencies in using MSMQ. Please share your experiences and/or suggestions in the comments!
Husband, father, and consultant who dabs mostly in Microsoft technologies. Loves tennis - both watching and playing.