Replicating Messages Between Instances

AMPS has the ability to replicate messages to downstream AMPS instances once those messages are stored to a transaction log. Replication in AMPS involves the configuration of two or more instances designed to share some or all of the published messages. With AMPS replication, an upstream (or source) instance delivers messages to a downstream instance.

Replication is typically used to improve the availability of a set of AMPS instances by creating a set of servers that hold the same messages, where each server can take over for the others in the event of a network or server failure.

Since AMPS replication operates on individual messages, replication is also an efficient way to split and share message streams between multiple sites where each downstream site may only want a subset of the messages from the upstream instances.

AMPS replication uses a leaderless, "all nodes hot" model. Any instance in a replication fabric can accept publishes and updates at any time, and there is no need for a message to be replicated downstream before it is delivered to subscribers (unless a subscriber explicitly requests otherwise using the fully_durable option on a bookmark replay).

The only communication between instances of AMPS is through replication. AMPS instances do not share state through the filesystem or any out-of-band communication.

AMPS high availability and replication do not rely on a quorum, controller instance, or leader instance. Each instance of AMPS processes messages independently. Each instance of AMPS manages connections and subscriptions locally.

AMPS supports two forms of message acknowledgment for replication links: synchronous and asynchronous; these settings control when AMPS considers a message persisted. This controls when AMPS sends publishers persisted acknowledgments and the point at which AMPS considers a message to be fully persisted for bookmark subscribers. These settings do not affect when or how messages are replicated, or when or how messages are delivered to subscribers unless a subscriber explicitly requests this behavior. These settings only affect when AMPS acknowledges to the publisher that the message has been persisted. They do not affect the speed of replication, the priority of replication, or the guarantees that AMPS makes for ensuring that all replicated messages are acknowledged by the downstream instance before they can be removed from the transaction log.

AMPS replication consists of a message stream (or, more precisely, a command stream) provided to downstream instances. AMPS replicates the messages produced as a result of publish and delta_publish and replicates sow_delete commands. AMPS does not replicate messages produced internally by AMPS, such as the results of Views or updates sent to a ConflatedTopic. When replicating queues, AMPS also uses the replication connection to send and receive administrative commands related to queues, as described in the section on Replicated Queues.

60East recommends that any server that participates in replication and that accepts publishes, SOW deletes, or queue acknowledgments directly from applications is configured with at least one sync replication destination.

Without at least one destination that uses sync acknowledgment, an AMPS instance could be a single point of failure, resulting in possible message loss if the instance has an unrecoverable failure (such as a hardware failure) between the time that the server acknowledges a command to a client, and the time that a replication destination receives and persists the command.

Notice that AMPS replicates fully processed and merged messages when replicating the results of a delta_publish command or a publish to a topic that provides preprocessing and enrichment. That is, AMPS replicates exactly the information that is written to the local transaction log. AMPS does not save the original command to the transaction log or replicate the original command. Instead, for topics recorded in the transaction log, AMPS stores the message produced by an update and replicates that complete message.