The way that data is stored and represented is an integral part of every data storage system. Whether it is a SQL or NoSQL database, a KV store, an AMQP / Streaming / Eventing system, a Graph database, or anything which stores data — control over the storage technology and technique is critical. This implementation of Raft uses the
RaftStorage trait to define the behavior needed of an application's storage layer to work with Raft.
There are a few important decisions which need to be made in order to implement the
- How do you plan on storing your snapshots? The
RaftStorage::Snapshotassociated type must declare the type your application uses for dealing with the raw bytes of a snapshot. For most applications, it stands to reason that a simple on-disk file is what will be used. As such, take a look at Tokio's fs::File. It satisfies all of the trait bounds for the
- How do you plan on storing your data? A majority of the methods of your
RaftStorageimpl will involve reading and writing data. Rust has a few data storage crates available to choose from which will satisfy these requirements. Have a look at Sled, or RocksDB. There are others to choose from, but these may be a solid starting point. Or you could always roll your own.
Once you're ready to begin with your implementation, be sure to adhere to the documentation of the
RaftStorage methods themselves. There are plenty of data safety requirements to uphold in order for your application to work properly overall, and to work properly with Raft.
For inspiration, have a look at this repo's
memstore project. It is an in-memory implementation of the
RaftStorage trait, intended for demo and testing purposes.
This implementation of Raft automatically triggers log compaction based on runtime configuration, using the
RaftStorage::do_log_compaction method. Everything related to compaction / snapshots starts with this method. Though snapshots are originally created in the
RaftStorage::do_log_compaction method, the Raft cluster leader may stream a snapshot over to other nodes if the node is new and needs to be brought up-to-speed, or if a node is lagging behind. Internally, Raft uses the
RaftStorage::Snapshot associated type to work with the snapshot locally and for streaming to follower nodes.
Compaction / snapshotting are not optional in this system. It is an integral component of the Raft spec, and
RaftStorage implementations should be careful to implement the compaction / snapshotting related methods carefully according to the trait's documentation.
When performing log compaction, the compaction can only cover the breadth of the log up to the last applied log and under write load this value may change quickly. As such, the storage implementation should export/checkpoint/snapshot its state machine, and then use the value of that export's last applied log as the metadata indicating the breadth of the log covered by the snapshot.
There is more to learn, so let's keep going. Time to learn about the most central API of this project.