Continuing from my previous post about Azure Service Bus, in this post I will share my learning about Queues & Messages. The focus of this post will be about some of the undocumented things I found as we implemented support for Queues and Messages in Cloud Portam.
Queues
As mentioned in my previous post, Queues is the simplest of the Azure Service Bus service and kind of compares with Azure Storage Queue Service in the sense that it provides a unidirectional messaging infrastructure where a publisher publishes a message and the message is received by a receiver. There can be many receivers ready to receive the messages however one receiver can only receive a message. No two receivers can receive a single message simultaneously.
Now some learning about Queues.
Queue Name
A queue name can be up to 260 characters in length and can contain letters, numbers, periods (.), hyphens (-), and underscores (_). A queue name is case-insensitive.
Queue Size
When creating a queue, you must define the size of the queue. Queue size could be one of the following values: 1 GB, 2 GB, 3 GB, 4 GB or 5 GB. A queue size can’t be changed once the queue is created. However if you create a “Partition Enabled Queue” then Service Bus creates 16 partitions thus your queue size is automatically multiplied by 16 and your queue size becomes 16 GB, 32 GB, 48 GB, 64 GB or 80 GB depending on the size you selected (This confused me initially :)).
Queue Properties
A Service Bus Queue has many properties. Some of the properties can only be set during queue creation time while some of the properties can only be set if you are using “Standard” tier of Service Bus.
(Above are the screenshots from Cloud Portam for creating a queue)
Status
Indicates the status of a queue – Active or Disabled. Once a queue is disabled, it cannot send or receive messages.
Max Delivery Count (MaxDeliveryCount)
Indicates the maximum number of times a message can be delivered. Once this count has exceeded, message will either be removed from the queue or dead-lettered. The way I understand it is this property is used to manage poison messages. If a message is not processed successfully by receivers for “x” number of times, just move it somewhere else for further inspection or remove it.
Message Time To Live (MessageTTL)
Indicates a time span for which a message will live inside a queue. If the message is not processed by that time, it will either be removed or dead-lettered. One interesting thing I noticed is that if you’re using “Standard” tier, a message could live forever in a queue however in “Basic” tier, a message can only live for a maximum of 14 days.
Lock Duration (LockDuration)
Indicates number of seconds for which a message will be locked by a receiver once it receives it so that no other receiver can receive that message. It essentially gives the receiver time to process the message. Once this elapses, message will be available to be received by another receiver. Maximum value for lock duration can be 5 minutes / 300 seconds.
Enable Partitioning (EnablePartitioning)
Indicates if the queue should be partitioned across multiple message brokers. As mentioned above, Service Bus automatically creates 16 partitions if this is enabled. This also results in maximum size of the queue increase by a factor of 16. This property can only be set during queue creation time.
Enable Deadlettering (EnableDeadLettering)
Indicates if the messages in the queue should be moved to dead-letter sub queue once they expire. If this property is not set, then the messages will be removed from the queue once they expire.
Enable Batching (EnableBatchedOperations)
Indicates if server-side batched operations are supported. This is used to improve the throughput of a queue as Service Bus holds the messages for up to 20ms before writing/deleting them in a batch.
Enable Message Ordering (SupportOrdering)
Indicates if the queue supports ordering.
Requires Duplicate Detection (RequiresDuplicateDetection)
Indicates if the queue requires duplicate detection. This property can only be set during queue creation time and is only available for “Standard” tier.
Enable Express (EnableExpress)
Indicates if the queue is an express queue. An express queue holds a message in memory temporarily before writing it to persistent storage. This property can only be set during queue creation time and is only available for “Standard” tier.
Requires Session (RequiresSession)
Indicates if the queue supports the concept of session. This property can only be set during queue creation time and is only available for “Standard” tier.
Auto Delete Queue
This property specifies a time period after which an idle queue should be deleted automatically by Service Bus. Minimum period allowed is 5 minutes. This can only be set for “Standard” tier.
Duplicate Detection History Time Window (DuplicateDetectionHistoryTimeWindow)
Defines the duration of the duplicate detection history. This can only be set for “Standard” tier.
Forward Messages To Queue/Topic (ForwardTo)
You can use this property to automatically forward messages from a queue to another queue or topic. When setting this property, the queue/topic must exist in the account. This can only be set for “Standard” tier.
Forward Dead-lettered Messages To Queue/Topic (ForwardDeadLetteredMessagesTo)
You can use this property to automatically forward dead-lettered message to another queue or topic. When setting this property, the queue/topic must exist in the account.
User Metadata (UserMetadata)
You can use this property to define any custom metadata for a queue.
Following table summarizes property applicability by tier and whether they are editable or not.
| Property | Tier | Editable? |
|---|---|---|
| Size | Basic, Standard | No |
| Status | Basic, Standard | Yes |
| Max Delivery Count | Basic, Standard | Yes |
| Message Time To Live | Basic, Standard | Yes |
| Lock Duration | Basic, Standard | Yes |
| Enable Partitioning | Basic, Standard | No |
| Enable Deadlettering | Basic, Standard | Yes |
| Enable Batching | Basic, Standard | Yes |
| Enable Message Ordering | Basic, Standard | Yes |
| Requires Duplicate Detection | Standard | No |
| Enable Express | Standard | No |
| Require Session | Standard | No |
| Auto Delete Queue | Standard | Yes |
| Duplicate Detection History Time Window | Standard | Yes |
| Forward Messages To Queue/Topic | Standard | Yes |
| Forward Dead-Lettered Messages To Queue/Topic | Basic, Standard | Yes |
| User Metadata | Basic, Standard | Yes |
To learn more about these properties, please see this link: https://msdn.microsoft.com/en-us/library/microsoft.servicebus.messaging.queuedescription.aspx.
Messages
The way I see it, messages are the entities that contain information about the work a sender wants a receiver to do. As mentioned earlier, a sender sends a message to a queue and a receiver will receive the message. At any time, a message will be received by one and only one receiver.
Message Processing
There’re two ways by which a receiver will receive a message: Peek and Lock & Receive and Delete.
Peek and Lock
In Peek and Lock mode, the message is locked by the receiver for a duration specified by Queue’s “Lock Duration” property or in other words under this mode a message is hidden from other receivers for a duration specified by Lock Duration. The receiver then would process the message and after that a receiver would mark the message as “Complete” which essentially deletes the message from the queue. If the “Lock Duration” expires, other receivers will be able to fetch this message.
Receive and Delete
In Receive and Delete mode, once the message is received by a receiver it will be deleted from the queue automatically. If a receiver fails to process that message, then the message is lost forever. So unless you’re sure of receiver’s functionality that it will never fail or you don’t care if the message is processed successfully or not, use this mode cautiously.
Message Composition
A message in Service Bus consists of 3 things – Message Body, Standard Properties and Custom Properties. Message Body is the actual content of the message. There are some predefined properties of a message and those fall under Standard Properties. Apart from that you can define custom properties on a message which are essentially a collection of name/value pairs. Total size of a message is 256 KB.
Message Properties
Now let’s take a look at some of the standard properties of a message that I found interesting.
Message Id
This is the identifier of a message. You can set it at the time of sending a message. Because it is an identifier, one would assume that it needs to be unique but that’s not the case. Different messages can have same Message Id.
Sequence Number
When a message is created, Service Bus assigns a number to a message. That number is stored in this property. Please note that it is a read-only property.
Message Time To Live (Message TTL)
This is the time period for which a message will remain in the queue. If you recall, you can also define a default message time-to-live at Queue level also. Service Bus actually picks the lower of the two values as message TTL. For example, if you have defined that a message will expire after 14 days at queue level but 5 minutes at the message level then the message will expire after 5 minutes.
Lock Token
Whenever a message is received by a receiver in “Peek and Lock” mode, Service Bus returns a (lock) token that must be used to perform further operations (e.g. delete message or dead-letter message etc.) on that message. This token is valid for a duration specified by “Lock Duration” property. After the lock duration expires, the lock token becomes invalid and any attempt to use this token for performing any allowed operations will result in an error. Once a lock token expires, a receiver must receive the message again.
There are other properties as well which I have not included for the sake of brevity. For a complete list of properties, please see this link: https://msdn.microsoft.com/en-us/library/microsoft.servicebus.messaging.brokeredmessage_properties.aspx.
Summary
That’s it for this post. In the next posts in this series, I will share my learning about Topics and other Service Bus services. So stay tuned for that! Again, if you think that I have provided some incorrect information, please let me know and I will fix them ASAP.
