Skip to main content

Bootstrap Initialization of AMPS Replication

AMPS offers the ability to initialize replication for an instance that has no existing state by recovering the full current set of messages from another instance.

Bootstrap initialization can restore the messages in the transaction log and current SOW of the source instance. Unlike replication, bootstrap initialization will restore messages that would not normally be replicated, including:

  • The full current contents of SOW topics (including messages that are present in the SOW but are no longer in the transaction log).

  • Messages that would not be replicated to the instance being bootstrapped due to protection against replication loops.

    That is, if the AMPS-Instance-A instance is being recovered, bootstrap recovery will copy messages that were originally published to AMPS-Instance-A.

    Replication does not return messages to the instance that the message was originally received from (to prevent loops, and because that instance is already known to have the message).

Bootstrap replication initialization uses the AMPS replication transport to retrieve messages from the source. However, the bootstrap initialization process itself is distinct from replication and happens as a part of AMPS recovery, before replication begins.

warning

Since an instance must finish bootstrap initialization before providing any other service, two instances of AMPS should not try to bootstrap from each other in a situation where both transaction logs are empty. In this case, both instances will wait for the other instance, and neither instance will recover.

If two instances of AMPS will be recovery partners with each other, start one instance without a bootstrap configuration the first time, then add the bootstrap configuration on a later restart.

Bootstrap initialization only takes place when the AMPS instance is starting, and the only function of bootstrap initialization is to restore the initial replicated state of AMPS from another instance. To keep current messages synchronized between instances, configure AMPS replication between the instances.

When to Use Bootstrap Initialization​

Bootstrap initialization is most commonly used for:

  • Disaster Recovery - In a situation where the entire state of the instance has been lost, bootstrap replication initialization can effectively recover the state of the instance from a stable disaster recovery instance, including the SOW topics and messages originally published to the instance.

  • Scale-out Scenarios - When an instance of AMPS joins an existing set of instances that maintain data in SOW topics for longer than the instances maintain the transaction log, bootstrap initialization can take the place of copying SOW files for providing current SOW data.

  • Offline Replica Scenarios - Bootstrap initialization can be a good option for periodically repopulating a development or reporting environment that periodically needs a full copy of the data in an instance but does not require active message flow.

    In this case, the state for the instance is typically removed on a regular basis. The message contents are recovered using bootstrap initialization, but the upstream instance might not be configured to continue to replicate messages while the offline replica instance is running, since active message flow is not necessary.

Starting Bootstrap Initialization​

An instance of AMPS will begin bootstrap initialization when:

  • The instance is configured to bootstrap the initial state.
  • The AMPS instance does not detect persistent state when it starts (or AMPS detects that it was shut down while bootstrap was underway).

An AMPS instance will only use bootstrap initialization when that instance starts with no messages in the transaction log. Bootstrap initialization will not overwrite or modify existing messages on an instance. If there are messages in the transaction log for the AMPS instance, the instance will not perform bootstrap initialization.

An AMPS instance does not use bootstrap initialization once the instance starts. To keep message state consistent between two instances of AMPS while AMPS is running, use AMPS replication.

Configuring Bootstrap Initialization​

To configure bootstrap initialization, the configuration on the instance being initialized must include:

  • A replication transport (that is, a Transport of type amps-replication or amps-replication-secure)
  • A transaction log, and
  • A bootstrap directive

Most often, the instance that is the source of bootstrap messages will also replicate to the instance that is being initialized. However, this is not required: the instance being bootstrapped can receive replication publishes from a completely different instance or set of instances.

The instance that will serve as the source of bootstrap initialization must define a replication transport and must also have recorded the information that will be used for initialization (that is, the topics that will be initialized must be in the transaction log and/or SOW).

For example, the following configuration will initialize the message state of the instance from the source AMPS instance located at source.example.com or, if that server is unavailable, at the AMPS instance located at backup.example.com. In both cases, those instances must have configured a replication transport listening on port 4100.

This instance will initialize the topics important-topic and any topic that begins with the string audit-topic of message type json from the source server. If either of these topics is recorded in the SOW, the current messages from the source instance will be copied to this instance of AMPS.

<Bootstrap>

  <Topic>
    <Name>important-topic</Name>
    <MessageType>json</MessageType>
  </Topic>

  <Topic>
    <Name>^audit-topic</Name>
    <MessageType>json</MessageType>
  </Topic>

  <DataType>sow,txlog</DataType>

  <Transport>
    <InetAddr>source.example.com:4100</InetAddr>
    <InetAddr>backup.example.com:4100</InetAddr>
    <Type>amps-replication</Type>
  </Transport>

</Bootstrap>

This instance must also define a replication transport and a transaction log. If the topics specified are to be stored in the SOW, the instance must also include definitions for those topics in the configuration.

If an instance with this configuration starts, and there is no message data in the transaction log, the instance will bootstrap initialize message contents as configured above.