Advanced Messaging and Queues

Queues are implemented as AMPS topics which lets you use the advanced messaging features of AMPS to create your queues and provide insight into your queues. For example, consumers can use content filtering to select the messages from the queue that they want to consume. You can use content filters to select only a subset of messages published to an underlying topic to populate the queue. You can even create a view that aggregates data from multiple topics, and use that view as the underlying topic for the queue. Since messages for queues are recorded in the transaction log, you can easily replay the messages that the queue provided to subscribers by using a bookmark subscription.

Querying Queues as a View

For each queue, AMPS provides a view of the currently available messages. Applications can query this queue just as though it were a view. For example, if you have a queue named PendingOrders, you can see the currently available messages in the queue by querying the queue as though it were a view, with a sow command.

A query of a queue is read-only. AMPS does not lease the returned messages to the querying application, or remove them from the queue.

Topic in the State of the World as an Underlying Topic for a Queue

AMPS fully supports a topic that maintains a SOW as an underlying topic for a queue. Since a queue records every individual publish to a topic (rather than simply preserving the current state of a distinct message identified by a SOW key), each publish to the SOW topic creates a new message in the queue.

AMPS does not provide Out-of-Focus messages to the queue. Only publish messages are added to the queue.

Deleting a message from an underlying topic that maintains a SOW does not remove the corresponding messages from the queue. Likewise, when a message expires from the SOW, it is not removed from the queue.

When a message is published to a topic in the SOW, and that topic uses expiration, the message is stamped with the expiration value set for the topic before being written to the transaction log. This means that the message will have an expiration timestamp that can be used by the queue for expiration. The ExpirationModel parameter of the queue configuration determines how the queue manages expiration when a message has an expiration set.

Delta Messaging with Queues

AMPS delta subscriptions rely on being able to determine the last state of a message delivered on a subscription and providing a set of changes to the subscriber. With AMPS queues, AMPS treats each update to a SOW record as a new message, so there's no previous state that would generate a delta message. When an underlying topic of a queue is a SOW topic, AMPS supports delta publish to that underlying topic. The full, merged message is added to the queue.

AMPS allows delta subscriptions to a queue, but treats each message as a new publish and delivers the full message.

Views and Aggregated Subscriptions over Queues

AMPS fully supports creating a view or an aggregated subscription with a queue as an underlying topic. In both cases, AMPS operates on the messages that are currently available in the queue. When a message is leased, that message is no longer available to the queue and does not appear in the view or the aggregated subscription. If the message is returned to the queue, then the message is again available to the view or aggregated subscription. When a message expires, that message is no longer available in the view or aggregated subscription.

Views and aggregated subscriptions are considered to be a query of the queue, so they are read-only. Views and aggregated subscriptions do not lease messages from the queue and do not affect message delivery.

Views over queues can be useful to show constantly-updated aggregates of the activity in the queue. For example, you could create an aggregate that shows the total value of unprocessed orders currently in the queue.

Bookmark Subscriptions to Queues

A queue itself supports at_most_once or at_least_once delivery. It does not provide a way to replay messages once a subscriber has acknowledged the message.

To support replay of messages from a queue, AMPS translates a bookmark subscription to a queue to be a bookmark subscription to the underlying topic for the queue. This allows you to replay messages from the underlying topic without queue delivery semantics. That is, a bookmark subscription to a queue becomes a publish/subscribe bookmark subscription to the underlying topic, not a subscription to the queue. Messages from this subscription do not have at-most-once or at-least-once delivery, and do not need to be acknowledged. The subscription is a publish/subscribe bookmark subscription, just as though there was no queue for the topic.

To get queuing semantics, do not include a bookmark on subscriptions to a queue.