Skip to content

Microsoft Message Queuing

August 10, 2009
Message Queuing
Message Queuing (also known as MQ) is a messaging infrastructure and a development tool for creating distributed messaging applications for Microsoft Windows operating systems. MSMQ makes it easy for application developers to communicate with application programs quickly, reliably, and asynchronously by sending and receiving messages.
MSMQ features
1. Guaranteed delivery
2. Message prioritization
3. Connectionless messaging
4. Multiple platform support
5. Performance
6. Asynchronous delivery
7. Notification services, which informs the sender that the message was received and processed.
8. Journaling, which maintains copies of the messages moving in the system.
9. Transactional Messaging
10. Secured Messaging
MQM can be successful where other forms of communication could not because it satisfies four important conditions:
  • No simultaneous connection is required between sender and receiver.
  • There are extremely strong request and response delivery guarantees even when communication does not occur simultaneously between sender and receiver.
ᅡᄋ Requests and responses can be translated and reformatted en route between senders and receivers.
MQM and Distributed Applications
The best way to understand the benefits of Message Queuing Middleware (MQM) may be to examine MQM in the context of a series of scenarios:
1. Store-and-forward communication
2. Defensive communication
3. Concurrent execution
4. Journaled communication
5. Connectionless communication.
Store-and-Forward Communication
MQM enables applications to send requests to other applications that are not expected to be running or reachable at the same time. Many applications run only at night, but must receive requests from applications that run only during the day.
By sending requests via MQM products, applications are assured that messages will be delivered as soon as network connections:
ᅡᄋ Become available.
ᅡᄋ Receiving applications begin processing.
Defensive Communication
In environments connected by local area networks, communication between applications is usually reliable. Nevertheless, any communications failure, no matter how infrequent, can cause serious problems (and WANs are notoriously more prone to failure than LANs).
By sending requests as MQM messages, applications will:
ᅡᄋ Be protected against communication losses when networks fail.
ᅡᄋ Be tolerant of normal peaks and valleys of demand.
ᅡᄋ Demonstrate excellent performance when networks are working properly.
Concurrent Execution
One of the challenges of using tightly coupled communication technology is making requests to more than one receiving application at a time. By definition, requestors using tightly coupled mechanisms must wait for the receiver to return a response before they can make a request to a different receiver. Methods used by developers to issue multiple synchronous calls at once are available, albeit requiring sophisticated (and expensive) programming techniques, such as using threads.
With MQM, applications are able to:
ᅡᄋ Send requests to many different receivers without waiting for responses.
ᅡᄋ Wait for the receivers to process the requests in parallel.
ᅡᄋ Process results when all of the response messages have arrived, or whenever is convenient.
Using MQM, applications can send their requests as messages and move on immediately to other tasks. This style of communication¬タヤoften called “fire-and-forget” message queuing¬タヤis truly difficult to implement with communications technologies that require the sending thread to wait for a response. With MQM, no special programming techniques are required. Development costs¬タヤand complexity¬タヤare contained.
Journaled Communication
Most mission critical environments require the ability to create journals of all communications activity within an application. Journals contain precise records that can be used by administrators for logging and audit purposes. Journals are also useful for error recovery; the journals are used to restore system state by replaying all events that occurred after a given point in time against a known starting point.
Connectionless Communication
Communication is said to be connection-oriented when an application must
ᅡᄋ Direct a request at a given instance of a receiver.
ᅡᄋ Wait for a response.
ᅡᄋ Share state information between the two applications for the duration of a call.
Direct RM
1. Direct RM supports conversations between two endpoints that assume close and dedicated interaction with each other.
2. The remote service must be reachable, up, and responsive for forward progress to be made, just like a phone conversation.
3. Services involved in the distributed application make forward progress and recover as a unit with a synchronized distributed state.
4. A direct RM session cannot last across service failures unless specific action is taken. When system failures occur, the services can recover by either of the following:
ᅡᄋ Application-specific messages can be used by a service to tell other services to go back to a mutually known earlier point.
ᅡᄋ Distributed transactions between the sender and the receiver can be used to simplify application code for recovery. A distributed two-phase commit automatically takes the state of all involved services back to the beginning of the transaction where the result of all messages sent and received is undone.
5. As illustrated in the above figure, direct RM offers the following guarantees that some transport protocols do not offer:
ᅡᄋ Exactly-once: This ensures that each message is delivered exactly-once.
ᅡᄋ In-order: This ensures that the order that the messages are sent by the RM Source application is the same order seen by the RM Destination application.
6. Direct RM has the following advantages over transport-level protocols:
ᅡᄋ End-to-end: This provides end-to-end instead of point-to-point guarantees. That is, when an ack is received, the sender knows the ultimate receiver has received the message instead of just the next intermediary (such as a firewall, network router, or load balancer). The ultimate receiver can sign the ack to prove it sent the ack ensuring message-level security.
ᅡᄋ Connection robustness: Direct RM provides a logical connection to the application that survives as underlying transport connections come and go.
ᅡᄋ Transport independent: As with other SOAP-level mechanisms, direct RM provides transport independence, so the same APIs are used across a variety of transports. This allows the transport to be changed without changing the application code.


Queued RM
1. Queued RM supports conversation between two endpoints that cannot assume tight interaction.
2. The remote endpoint does NOT have to be reachable, up, or responsive for the local endpoint to make forward progress.
3. Applications using queued RM use smaller local units of forward progress and recovery than applications using direct RM.
4. The arrival of an input message just causes the message to be enqueued, adding to the to-do work for the service.
5. When resources are available to process the input message, a state transition uses a local transaction to dequeue the message, process it, and enqueue output messages, all without involving remote endpoints.
6. After commit, the RM infrastructure begins its attempt to transmit the output messages, only requiring the remote service to eventually become available.
7. These smaller local transacted steps provide the isolation needed to deal with unpredictable responsiveness in remote services.
8. In addition to the advantages of direct RM, queued RM offers the following advantages:
ᅡᄋ Independent progress: As described above, when a remote service is not available, a service can make progress locally. A state transition (forward or backward) takes place without communicating with other services.
ᅡᄋ Fixed cost per message: It can be dangerous to hold expensive resources (for example, memory, threads, data locks) waiting for the next input message. Instead, it is safer for each service in the distributed application to give up expensive resources (deactivate), and reacquire them when the next input message arrives (activate).
ᅡᄋ Recovery from service failure: Some remote endpoints may be highly reliable and available (e.g., a server farm with a clustered database). Others may be periodically out of service (for example, a personal machine or small single-machine server). A queued RM session can last across service failures, because RM metadata (sequence information, messages, etc.) is stored in a transacted durable store that can safely store output messages for later transmission and input messages for later delivery after recovery.
ᅡᄋ The storage used by queued RM is a transactional resource manager, so that output messages can be enqueued and input messages can be dequeued as part of an application transaction
ᅡᄋ Since enqueue (for send) does not actually happen until the transaction commits, transmission does not begin until after commit, so an abort automatically “takes back” the messages the application sent during the transaction.
ᅡᄋ Since dequeue (for delivery) does not actually happen until the transaction commits, an abort leaves the message in the input queue so it can be retried later in another transaction.
The tables below furnish the performance statistics between COM component services and .Net WebServices
IPersistStream vs. .NET Serialization (Send)
Msg Per Sec
StdDev
Mode
Local/Remote
.NET
COM
.NET Optimized
.NET
COM
.NET Optimized
Express
Local
1,718
6,188
13,305
5.34
331.93
685.93
Remote
1,441
4,523
9,129
3.29
44.34
247.91
Recoverable
Local
910
1,490
2,236
26.65
57.45
43.44
Remote
948
1,333
1,973
3.89
46.42
16.59
Transactional
Local
791
1,261
1,660
18.1
13.26
81.88
Remote
603
1,114
954
15.81
295.88
29.2
IPersistStream vs. .NET Serialization (Receive)
Msg Per Sec
StdDev
Mode
Local/Remote
.NET
COM
.NET Optimized
.NET
COM
.NET Optimized
Express
Local
1,870
3,049
10,441
1.9
10.59
16.65
Remote
509
511
672
2.34
2.34
1.64
Recoverable
Local
1,096
1,373
2,042
38.58
27.65
118.32
Remote
505
528
690
4.18
2.88
1.83
Transactional
Local
922
1,203
1,604
19.77
5.68
71.65
.NET Formatter Comparison (Enqueuing)
Msg Per Sec
StdDev
Mode
Local/Remote
.NET Binary
.NET XML
.NET Binary
.NET XML
Express
Local
1,718
1,344
5.34
15.38
Remote
1,441
987
3.29
4.22
Recoverable
Local
910
573
26.65
7.33
Remote
948
682
3.89
5.59
Transactional
Local
791
493
18.1
5.68
Remote
603
495
15.81
13.99
.NET Formatter Comparison (Dequeuing)
Msg Per Sec
StdDev
Mode
Local/Remote
.NET Binary
.NET XML
.NET Binary
.NET XML
Express
Local
1,870
791
1.9
1.48
Remote
509
282
2.34
1.14
Recoverable
Local
1,096
594
38.58
7.19
Remote
505
284
4.18
0.71
Transactional
Local
922
533
19.77
6.44
Advertisements
No comments yet

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: