The recent Tendermint Core 0.34 release includes support for state sync, which allows a new node to join a network by fetching a snapshot of the application state at a recent height instead of fetching and replaying all historical blocks. This can reduce the time needed to sync with the network from days to minutes.

A Tendermint blog post outlined the state sync protocol itself, and more details are available in the ABCI documentation. However, the good news is that the upcoming Cosmos SDK 0.40 release will include automatic support for state sync, and developers only need to enable it in their applications to make use of it. We’ll explain how in this article, but first some background.

Tendermint Core handles most of the grunt work of discovering, exchanging, and verifying state data for state sync, but the application must take snapshots of its state at regular intervals and make these available to Tendermint via ABCI calls, and be able to restore these when syncing a new node.

The Cosmos SDK stores application state in a data store called IAVL, and each module can set up its own IAVL stores. At regular height intervals (which are configurable), the Cosmos SDK will export the contents of each store at that height, Protobuf-encode and compress it, and save it to a snapshot store in the local filesystem. Since IAVL keeps historical versions of data, these snapshots can be generated simultaneously with new blocks being executed. These snapshots will then be fetched by Tendermint via ABCI when a new node is state syncing.

Note that only IAVL stores that are managed by the Cosmos SDK can be snapshotted. If the application stores additional data in external data stores, there is currently no mechanism to include these in state sync snapshots, so the application therefore cannot make use of automatic state sync via the SDK. However, it is free to implement the state sync protocol itself as described in the ABCI documentation. Support for this may be added in a later version.

When a new node is state synced, Tendermint will fetch a snapshot from peers in the network and provide it to the local (empty) application, which will import it into its IAVL stores. Tendermint then verifies the application’s app hash against the main blockchain using light client verification, and proceeds to execute blocks as usual. Note that a state synced node will only restore the application state for the height the snapshot was taken at, and will not contain historical data nor historical blocks.

To enable state sync snapshots, an application using the Cosmos SDK BaseApp needs to set up a snapshot store (with a database and filesystem directory) and configure the snapshotting interval and the number of historical snapshots to keep. A minimal example of this might be as follows:

When starting the application with appropriate flags, e.g. --state-sync.snapshot-interval 1000 --state-sync.snapshot-keep-recent 2, it should generate snapshots and output log messages about it:

Note that the snapshot interval must currently be a multiple of the pruning-keep-every(default 100), to prevent heights from being pruned while taking snapshots. It’s also usually a good idea to keep at least 2 recent snapshots, such that the previous snapshot isn’t removed while a node is attempting to state sync using it.

Once a few nodes in a network have taken state sync snapshots, new nodes can join the network using state sync. To do this, the node should first be configured as usual, and the following pieces of information must be obtained for light client verification:

This can be obtained e.g. via RPC (must be from a trusted source):

We can then configure Tendermint to use state sync in config.toml:

When the node is started it will then attempt to find a state sync snapshot in the network and restore it:

And that’s it, the node is now state synced, having joined the network in seconds.

Cosmos SDK State Sync Guide was originally published in Cosmos Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

314 +