[Follower ] Backup / restore follower DB

[musitdev]

Summary

• RFCs: Link to RFC, Link to RFC, or $\emptyset$.
• Categories: any of protocol-units, networks, scripts, util, cicd, or misc.
  Add backup/restore command to the follower node

Changelog

Move the chunk part in the S3 part because it's more logic. The chunks are needed by S3 communication.
Add docker files to backup / restore/pull.
Add the follower backup docker script; it still needs to be tested—no need for a follower node.

Testing

First, run a local node to create a DB in .movement.

Build the full node exe: cargo build -p movement-full-node

Unit Test:

RUST_BACKTRACE=1 AWS_ACCESS_KEY_ID=<key> AWS_SECRET_ACCESS_KEY="<key>" AWS_REGION=us-west-1 cargo test -p syncador test_archive_split -- --nocapture

Local backup command

DOT_MOVEMENT_PATH="$(pwd)/.movement" AWS_REGION=us-west-1 AWS_ACCESS_KEY_ID="<key>" AWS_SECRET_ACCESS_KEY="<key>" target/debug/movement-full-node backup save-and-push

Local retore command

DOT_MOVEMENT_PATH="$(pwd)/.movement" AWS_REGION=us-west-1 AWS_ACCESS_KEY_ID="<key>" AWS_SECRET_ACCESS_KEY="<key>" target/debug/movement-full-node backup restore

Restart the node which should start correctly.

Outstanding issues

The backup follower node docker script hasn't been tested. Currently, there's an issue with the sh command that is not seen. Short term we can do a backup manually.
Partner follower node are not concerned by this issue.

[andygolay]

I ran all three tests locally. They all pass.

I have not reviewed the code in much detail but I looked through the overlays and they seem to make sense.