Handling Unprocessed Messages

Queuing applications occasionally encounter data that cannot be successfully processed or completed. AMPS allows you to configure actions that occur when a message expires from a message queue. You can create a dead-letter queue or topic by using the amps-action-on-sow-expire-message action in conjunction with the MaxCancels and MaxDeliveries options and the cancel and expire options on the sow_delete command.

As an example, suppose your queue Jobs is used to coordinate computation jobs, and in the rare case where a job cannot be completed successfully, you would like AMPS to automatically move the job to another queue, FailedJobs. AMPS provides MaxCancels and MaxDeliveries options to specify how many attempts it should make to deliver a message to a subscriber:

<AMPSConfig>
    ...

    <SOW>
        <Queue>
            <Name>Jobs</Name>
            <MessageType>json</MessageType>
            <MaxCancels>3</MaxCancels>
            <MaxDeliveries>10</MaxDeliveries>
        </Queue>
        <Queue>
            <Name>FailedJobs</Name>
            <MessageType>json</MessageType>
        </Queue>
    </SOW>
</AMPSConfig>

In this configuration, AMPS will automatically expire a message from Jobs on the 3rd time it receives a sow_delete with the cancel operation for that message, or after the 10th unsuccessful delivery of the message. By default, expiring a message simply means the message is removed from the queue, but in this case, we also want the FailedJobs queue to receive the message so that it can be manually reviewed later. To accomplish this, AMPS provides the amps-action-on-sow-expire-message action which can be configured to do so:

<AMPSConfig>
    ...
    <Actions>
        <Action>
            <On>
                <Module>amps-action-on-sow-expire-message</Module>
                <Options>
                    <Topic>Jobs</Topic>
                    <MessageType>json</MessageType>
                </Options>
            </On>
            <Do>
                <Module>amps-action-do-publish-message</Module>
                <Options>
                    <Topic>FailedJobs</Topic>
                    <MessageType>json</MessageType>
                    <Data>{ "message": {{AMPS_DATA}}, "reason": "{{AMPS_REASON}}" }</Data>
                </Options>
            </Do>
        </Action>
    </Actions>
    ...
</AMPSConfig>

With this action configured, any expired message from Jobs will result in a message automatically published to FailedJobs containing the original message inside the message element, and an AMPS-supplied reason code in the reason field.