Out-of-Focus Messages (OOF)
One of the more difficult problems in messaging is knowing when a record that previously matched a subscription has been updated so that the record no longer matches the subscription. AMPS solves this problem by providing an out-of-focus, or OOF, message to let subscribers know that a record they have previously received no longer matches the subscription. The OOF messages help subscribers easily maintain state and remove records that are no longer relevant.
OOF notification is optional. A subscriber must explicitly request that AMPS provide out-of-focus messages for a subscription.
When OOF notification has been requested, AMPS produces an oof message for any record that has previously been received by the subscription at the point at which:
The record is deleted
The record expires
The record no longer matches the filter criteria
The record is no longer within the pagination window (for paginated subscriptions), or
The subscriber is no longer entitled to view the new state of the record.
AMPS produces an oof message for each record that no longer matches the subscription. The oof message is sent as part of processing the update that caused the record to no longer match. Each oof message contains information the subscriber can use to identify the record that has gone out of focus and the reason that the record is now out of focus.
Since AMPS must maintain the current state of a record to know when to produce an oof message, these messages are only supported for SOW topics, conflated topics, and views. The oof option is not supported for subscriptions that do not include a SOW query or bookmark replays.
When AMPS returns an OOF message, the data contained in the body of the message represents the updated state of the message (except as described below). This will allow the client to make a determination as to how to handle the data, be it to remove the data from the client view or to change their query to broaden the filter thresholds. This enables a client to take a different action depending on why the message no longer matches. For example, an application may present a different icon for an order that moves to a status of completed than it would present for an order that moves to a status of cancelled.
When a delta_publish message causes the SOW record to go out of focus, AMPS returns the merged record.
When there is no updated message to send, AMPS sends the state of the record before the change that produced the oof. This can occur when the message had been deleted, when the message has expired, or when an update causes the client to no longer have permission to receive the record.
For a conflated view or a subscription that uses conflation, the data included in the oof message will be the last data that the subscriber received. When both an update to a message and a change that would cause the message to go out of focus happen in the same conflation interval, the subscriber receives an oof notification with the previously-received state of the message. Likewise, if a change that causes a message to go out of focus and a change that causes the message to come back into focus occur within the same conflation interval, the subscriber receives the state of the message at the end of the conflation interval. The subscriber does not receive an indication that the message had gone out of focus during the conflation interval and then come back into focus.
Out-of-Focus Reason Codes
An out of focus message returns the reason that the message was produced in the reason field of the message header.
deleted
The message was deleted from the topic.
previous message
expired
The message was removed from the topic due to expiration.
previous message
match
The message no longer matches the content filter or has moved outside of the record set requested.
updated message that no longer matches subscription criteria
entitlement
The message has changed such that the user is not entitled to see the updated message.
previous message
Usage
Consider the following scenario where AMPS is configured with the following SOW key for the buyer topic:
When the following message is published, it is persisted in the SOW topic:
A client issues a sow_and_subscribe request for the topic buyer with the filter /buyer/loc="NY" and the oof option set on the request. The client will be sent the message as part of the SOW query result.
Subsequently, the following message is published to update the loc tag to LN:
The original message in the SOW cache is updated. The client does not receive the second publish message, because that message does not match the filter (/buyer/loc="NY"). This is problematic. The client has a message that is no longer in the SOW cache and that no longer matches the current state of the record. Since the oof option was set on the subscription, however, the AMPS engine sends an oof message to let these clients know that the message that they hold is no longer in the SOW cache. The following is an example of what's returned.
The header of the message will contain the following fields to help the application identify the reason for the oof message and which message no longer matches:
Command
oof
Topic
buyer
Reason
match
SowKey
6387219447538349146
The message data will contain the updated message, as shown following.
Had the message been deleted, the message data for the OOF notification would contain deleted message, and the reason would be deleted.
An easy way to think about the situations where AMPS sends an OOF notification is to consider what would happen if the client re-issued the original sow request after the above message was published. The /client/loc="NY" expression no longer matches the message in the SOW cache and as a result, this message would not be returned.
Example
To help reinforce the concept of OOF messages, and how OOF messaging can be used in AMPS, consider a scenario where there is a GUI application whose requirement is to display all open orders of a client. There are several possible solutions to ensure that the GUI client data is constantly updated as information changes, some of which are examined below; however, the goal of this section is to build up a sow_and_subscribe message to demonstrate the power that OOF notifications add to AMPS.
Client-Side Filtering in a sow_and_subscribe Command
First, consider an approach that sends a sow_and_subscribe message on the topic orders using the filter /Client="Adam".
AMPS completes the sow portion of this call by sending all matching messages from the orders SOW topic. AMPS then places a subscription whereby all future messages that match the filter get sent to the subscribing GUI client.
As the messages come in, the GUI client will be responsible for determining the state of the order. It does this by examining the State field and determining if the state is equal to “Open” or not, and then updating the GUI based on the information returned.
This approach puts the burden of work on the GUI and, in a high volume environment, has the potential to make the client GUI unresponsive due to the potential load that this filtering can place on a CPU. If a client GUI becomes unresponsive, AMPS will queue the messages to ensure that the client is given the opportunity to catch up. The specifics of how AMPS handles slow clients is covered in the section discussing Slow Client Management.
AMPS Filtering in a sow_and_subscribe Command
The next step is to add an additional ’AND’ clause to the filter. In this scenario we can let AMPS do the filtering work that was previously handled on the client. This is accomplished by modifying our original sow_and_subscribe to use the following filter:
Similar to the above case, this sow_and_subscribe will first send all messages from the orders SOW topic that have a Client field matching "Adam" and a State field matching "Open". Once all of the SOW topic messages have been sent to the client, the subscription will ensure that all future messages matching the filter will be sent to the client.
There is a less obvious issue with this approach to maintaining the client state. The problem with this solution is that, while it initially will yield all open orders for client "Adam", this scenario is unable to stay in sync with the server. For example, when the order for Adam is filled, the State changes to State=Filled. This means that, inside AMPS, the order on the client will no longer match the initial filter criteria. The client will continue to display and maintain these out-of-sync records. Since the client is not subscribed to messages with a State of “Filled,” the GUI client would never be updated to reflect this change.
OOF Processing in a sow_and_subscribe Command
The final solution is to implement the same sow_and_subscribe query which was used in the first scenario. This time, we use the filter requests only for the State that we're interested in, but we add the oof option to the command so the subscriber receives OOF messages.
AMPS will respond immediately with the query results, exactly as it does with a sow_and_subscribe command that does not use the oof option.
This approach provides the following advantage: for all future messages in which the same Open order is updated, such that its status is no longer Open, AMPS will send the client an OOF message specifying that the record which previously matched the filter criteria has fallen out of focus. AMPS will not send any further information about the message unless another incoming AMPS message causes that message to come back into focus.
In the following diagram, the Publisher publishes a message stating that Adam’s order for MSFT has been fulfilled. When AMPS processes this message, it will notify the GUI client with an oof message that the original record no longer matches the filter criteria. The oof message will include a Reason field with it in the message header, defining the reason for the message to lose focus. In this case the Reason field will state match since the record no longer matches the filter.
AMPS will also send oof messages when a message is deleted or has expired from the SOW topic.
We see the power of the oof message when a client application wants to have a local cache that is a subset of the SOW. This is best managed by first issuing a query filter sow_and_subscribe which populates the GUI, and enabling the oof option. AMPS informs our application when those records which originally matched no longer do, at which time the program can remove them.
Last updated
