SOW/*Queue
SOW/Queue, SOW/LocalQueue, and SOW/GroupLocalQueue
This section lists configuration parameters for queues.
The Queue
tag, the LocalQueue
tag and the GroupLocalQueue
tag are used to configure message queues. The replication model of the queue is specified by the exact tag used.
Defining Queue Replication Type
The tag that encloses a queue definition specifies the queue replication type. Full details are in the Queue Replication Types section in the AMPS User Guide.
To summarize:
When an AMPS queue is defined with the
Queue
tag, the queue will be a distributed queue. The queue may be consumed from any instance that hosts the queue, and all instances that accept publishes for queue messages must host the queue.To make a queue that is limited to the local instance only, use the
LocalQueue
configuration element. This makes a queue independent of any other instance.To make a queue that is restricted to a subset of instances, in a set of replicated instances (typically all in the same group), use the
GroupLocalQueue
tag. This creates a queue intended to be hosted by some, but not all, of the instances in the replicated fabric host the queue.
AMPS accepts QueueDefinition
as a synonym for Queue
.
Queue Configuration
The following configuration items apply to queues regardless of whether they are fully distributed, local, or distributed within a replication group:
Element | Description |
| The name of the queue topic. This name is the name that consumers subscribe to.
If no |
| The message type of the queue. |
| A topic name or regular expression for the topic that contains the messages to capture in the queue. These topics must be recorded in a transaction log, and all must be of the same message type as the queue.
Default: The |
| The topic to publish to when an application publishes a message to the queue. For simplicity, AMPS allows applications to publish messages to the queue, and for those messages to be routed to one of the underlying topics.
The |
| The amount of time that a subscriber has ownership of the message before the message is returned to the queue.
For |
| The delivery semantics to use for this queue. Regardless of the delivery semantics, AMPS queues deliver a given message to a single subscriber at a time. When a subscriber fails to process a message (that is, the connection to the subscriber closes before the message is acknowledged, or the lease expires before the message is acknowledged), the semantics specify how the failure is handled.
Default: |
| The maximum number of outstanding, unacknowledged messages in the queue at any one time. This parameter allows you to set limits on the number of pending messages from the queue overall. When the queue reaches the Notice that this does not set a limit of any sort on the capacity of the queue. This parameter allows you to limit the number of messages that the queue will make available to subscribers at a given time, but does not restrict the capacity of the queue to track messages. This backlog number is applied per instance of the queue. That is, each instance of AMPS that hosts an instance of a replicated queue will deliver messages up to the Default: unset (no limit) |
| The maximum number of outstanding, unacknowledged messages in the queue for an individual subscription. This parameter allows you to avoid overwhelming a single subscriber during a period of heavy activity. Subscribers can declare the maximum number of messages that the subscription is prepared to lease at a given time. This maximum defaults to Notice that this does not set a limit of any sort on the capacity of the queue. This parameter allows you to limit the number of messages that the queue will make available for a single subscription at a given time, but does not restrict the capacity of the queue to track messages.
Default: |
| Sets the queue default value for the length of time an individual message can remain in the queue before AMPS considers the message to be undeliverable. Messages may expire while a subscriber has a lease on the message. AMPS does not send an additional notification in this case. It is possible for a message to also have an expiration value assigned. In this case, the |
| Manages how AMPS applies the queue expiration period and the message expiration period to determine the expiration for the message. There are four models:
Notice that an
Default: |
| An AMPS By default, there is no filter and all messages from the |
| This option allows you to specify the point at which AMPS begins reviewing the transaction log to recover the state of the queue when AMPS restarts and there is no existing information about the state of the queue. By default, in this case AMPS reviews the full log to determine the contents and state of the queue. If AMPS has a record of the last point in the transaction log at which all previous messages are acknowledged (stored in the The
Default: |
| AMPS provides different methods to distribute messages across active subscriptions:
Each instance of AMPS independently manages the fairness model for subscriptions on that instance. Fairness model information is not replicated across instances. Default: |
| Ownership model for leased messages. AMPS supports the following models:
Default: |
| Specifies an upper bound to the number of times AMPS may deliver a queue message before automatically expiring it. For example if AMPS delivers a message twice and This counter is reset if the server restarts, and the counter is not replicated to other instances. Default: No maximum ( |
| Specifies a limit to the number of times a subscriber may cancel a lease on a message before it is expired.
For example if a message is canceled for the second time and Default: No maximum ( |
| Specifies the order in which messages will be distributed from the queue. When present, this element constructs a value that specifies the priority for messages in the queue. Higher priority messages are delivered first, regardless of the order in which messages have been published to the queue. The contents of the element can be either the name of a field in the message or an AMPS expression. Either way, the result is treated as an For example, to order message delivery based on the To order message delivery based on the product of the Default: There is no default for this value. When not provided, the delivery order is the order in which messages were processed by this instance of AMPS. This option cannot be specified on a queue if |
| Specifies the filter used to identify a barrier message (synchronization point) for this queue. When a message matches this expression, it will be delivered to all current subscribers on the queue when every previous message in the queue has been acknowledged. This provides a simple, reliable way to synchronize workers.
For example, the following configuration item specifies that any message that contains a non-NULL The Default: There is no default for this value. When not provided, no messages are considered to be barrier messages. This option cannot be specified on a queue if |
Managing Queue Metadata
AMPS provides the following options for managing queue metadata:
Element | Description |
| Specifies whether AMPS should persist metadata about the queue in the journal directory. This could reduce the active memory footprint of an AMPS instance in cases where a queue has a large number of messages, but it is not being actively consumed. In cases where the queue has a large number of unacknowledged messages when AMPS is restarted, this may also improve recovery time. This option can be unset or set to a value of Default: unset, which indicates that queue metadata will not be persisted |
| The target number of messages to keep active state on. Providing this option limits the amount of state that AMPS keeps for unacknowledged messages. In most circumstances, the limit will be the number of messages specified in this parameter. If this instance receives a transfer request for a message that is in the transaction log, but outside the currently active set of messages, AMPS will process the queue until it reaches the message to be transferred. Messages that are present in the transaction log, but for which AMPS does not currently have active state, are not deliverable from the queue topic, cannot be queried, cannot be deleted using a This option is not set by default. When this option is not set, all messages in the queue are active. This option is typically unnecessary if the This option cannot be set if When this parameter is set, it should be set to a number that accounts for several seconds of traffic to the queue, considering replication and acknowledgment speeds. For example, if a queue is typically processed at a rate of 2500 messages per second within minimal replication delays, a reasonable minimum target value might be 12500 or 15000. Notice that, if subscribers use content filters to selectively process the queue, if no currently active messages match the subscription, no messages will be delivered (even if there are matching messages in the queue that are not currently active). Also notice that this option may increase CPU cost, latency, and disk activity for queues, since metadata for messages is loaded as messages are processed rather than when messages arrive from a publisher. Default: unset (no limit) Minimum: |
| Specifies the amount of time for AMPS to retain information about an acknowledgment ( This element is configured as an interval, for example, The default value is generally recommended. However, in cases where an instance may receive large volumes of acknowledgments for messages that are not currently in the queue, and are not expected to arrive, setting this to a lower value may somewhat reduce the memory required for managing these acknowledgments. Default: |
Group Local Queue Configuration
When a queue is defined as a GroupLocalQueue
the following configuration item must be provided in addition to the mandatory elements above:
Element | Description |
| For a group local queue, provides the instance |
When a queue is defined as a GroupLocalQueue
, the following optional configuration element may be provided.
Element | Description |
| For a group local queue, provides a way to allow a queue that is hosted in instances that are in different replication groups to be identified as the same queue and function as a single distributed queue.
By default, a |
The following configuration snippet shows one way to configure a queue.
The following configuration snippet shows the configuration for a GroupLocalQueue
. Any instance of the queue in the group named CONSUME_INSTANCES
or that uses the GroupLocalQueueDomain
tag of CONSUME_INSTANCES
will be treated as an instance of the same queue.
Last updated