AMPS provides the ability for incoming commands to be modified, or filtered.
When one or more transport filters are specified, AMPS provides each incoming command to those filters as soon as the header for the message is parsed. Each filter can modify the message data or a subset of the headers, and can choose to have AMPS stop processing the command (or can request that AMPS disconnect the connection that submitted the command).
The filters for a Transport, if any, are defined in the configuration for the transport. When more than one filter is specified, AMPS runs each filter in the order in which they appear in the configuration file.
Transport filters are implemented as extension modules. To create an extension module, contact AMPS support for the server SDK.
AMPS loads the following transport filters by default. Expand each item for more details.
amps-topic-translator
Translates topic names on incoming commands.
This module requires one or more of the following options: Topic
Specifies the translation to use. This option takes the following format:
original:translated
The original parameter can be a literal topic name or a PCRE regex. Any topic on any command that matches that parameter will be converted to the translated topic.
For example, to convert the topic legacy to the topic new, you would specify the following option:
<Options> <Topic>legacy:new</Topic> </Options>
To translate any topic beginning with /orders/northamerica to NAorders, you would specify the following option:
Translates an incoming subscribe, sow_and_subscribe, delta_subscribe, or sow_and_delta_subscribe command for a specific topic name as follows:
Translates the topic name on the command to a different topic name.
Adds a conflation interval to the command, if there is no conflation interval specified on the incoming command.
This module can be useful for removing a conflated topic that is infrequently-used, or for which subscribers only monitor a small number of messages out of the overall topic.
This module requires one or more of the following options: Topic
Specifies the translation to use. This option takes the following format:
original:translated:interval
The interval specifies the conflation interval to apply to the translated commands if one is not provided.
For example, to convert all subscriptions to the topic orders-C to the topic orders, and guarantee that each translated subscription has a conflation specified, with a 500 millisecond default for conflation, you would use the following options:
To convert all subscriptions to the topics slowUpdates and verySlowUpdates to the topic updates and guarantee that each translated subscription has a conflation specified, with a 2 second default for conflation, you would use the following options:
The following transport filter is included in the AMPS distribution, but is not loaded by default. Expand the item for more details.
Correlation ID Timestamper
Writes an ISO 8601 format timestamp to the correlation ID for publish and delta_publish messages received by AMPS.
This module can be useful for gaining a better understanding of AMPS message latency by facilitating the latency measurement of a publish message from the point that AMPS processes the message to the point that it is received by a consumer. It’s important to note that the latency measurement is not end to end latency as it doesn’t include the latency between the publisher and AMPS (since the publisher does not timestamp the message).
To load the module in AMPS, add the following configuration item to the Modules block of the AMPS configuration file. Then, to use the module, add it to the TransportFilter section of the Transports block, as shown below.
<Modules>
<Module>
<Name>transport-filter-correlation-id-timestamper</Name>
<Library>libamps_transport_filter_correlation_id_timestamper.so</Library>
</Module>
</Modules>
<Transports>
<Transport>
<Name>primary</Name>
<Type>tcp</Type>
...
<TransportFilter>
<Module>transport-filter-correlation-id-timestamper</Module>
<Options>
<Override>True</Override> <!-- Override existing correlation ID if one exists -->
</Options>
</TransportFilter>
</Transport>
</Transports>
This module supports a single option - Override. If specified and set to True, the module will overwrite the correlation ID value if it is set. Otherwise, the value will not be overwritten.
Note: For certain features such as bookmark subscriptions, SOW topics and views, you may see unexpected results. For example, in the case of a bookmark subscription, messages could be replayed that were received by AMPS hours previous to the bookmark subscription and therefore not result in a meaningful latency measurement if compared with the current time. This is not caused by any delay in AMPS, but rather because the received messages are from historical data (this scenario would be similar for a SOW query).
In the case of views, the messages are generated internally by AMPS. Given this, transport filters do not apply and therefore timestamps are not added to the correlation ID field.
In the case of replication, replicated messages will not be restamped and therefore the timestamp in the correlation ID header field is not overwritten when a message is replicated.
Additionally, when calculating latency, it is critical that the clocks on the hosts being utilized are in sync. For example, if AMPS is running on host A and timestamping publishes, then messages are received on host B and the latency calculation is performed, if the clocks on host A and B are not in sync the latency measurement could be invalid and possibly even be a negative value.
