docs: update documentation to cover recent changes

Author: garethgeorge

docs/content/1.introduction/1.getting-started.md

@@ -20,9 +20,22 @@ See the <a href="https://github.com/garethgeorge/backrest" target="_blank">Githu
 ## Configuration
 
 ::alert{type="info"}
-Once backrest is installed, you can access the web interface at `http://localhost:9898` (or the port you configured). Backrest will immediately prompt you to configure a default user and password (or you may disable authentication if you prefer). Once this is done, you can go on to configure your installation.
+Once backrest is installed, you can access the web interface at `http://localhost:9898` (or the port you configured). Backrest will immediately prompt you for required initial configuration of an instance ID and a default user and password.
 ::
 
+Instance ID
+
+ * The instance ID is a unique identifier for your Backrest instance. This is used to tag snapshots created by Backrest so that you can distinguish them from snapshots created by other instances. This is useful if you have multiple Backrest instances backing up to the same repo.
+ * *Notably the instance ID cannot be changed after initial configuration as it is stored in your snapshots. Choose a value carefully.*
+
+Username and password
+
+ * Username and password is set on first launch of Backrest. If you lose your password you can reset it by deleting the `"users"` key from the `~/.config/backrest/config.json` file and restarting the Backrest service.
+ * If you don't want to use authentication (e.g. a local only installation or if you're using an authenticating reverse proxy) you can disabled authentication.
+
+
+#### Backrest Host Name
+
 #### Add a new repository
 
 A Backrest repository is implemented as a restic repository under-the-hood (more on this later). A Repo is a configuration object which identifies a storage location and the credentials that will be used to encrypt snapshots sent to that storage. You can either add an existing repo that you created on the restic CLI or create a new one in the Backrest UI. In either case, click the "Add Repo" button in the UI to configure Backrest to use your backup location.
@@ -53,6 +66,9 @@ The primary properties of a repository are:
 
 - **Hooks** are actions triggered by backup lifecycle events, repo hooks will apply to actions run by any plan that writes to the repo. Hooks can also be configured at the plan level. See the [hooks documentation](/docs/hooks) for more information.
 
+- **Prune Policy** the policy that determines when prune operations will run. Prune operations clean up unreferenced data in your repo (freeing up storage in your repo over time). See the [prune documentation](/docs/operations#prune) for more information.
+- **Check Policy** the policy that determines when check operations will run. Check operations periodically verify the integrity of your backups. See the [check documentation](/docs/operations#check) for more information.
+
 ::alert{type="info"}
 Once you've added your repo you're ready to import snapshots (click Index Snapshots) if you have existing data. Go on to configure a backup plan to start creating new snapshots.
 ::
@@ -79,7 +95,7 @@ The primary properties of a plan are:
   - **By Time Period** - keeps snapshots in a time-based bucket. E.g. keep 1 snapshot per day for the last 7 days, 1 snapshot per week for the last 4 weeks, and 1 snapshot per month for the last 12 months.
   - **None** - no retention policy is enforced. This is useful for append-only repos or if you'd like to manage retention manually e.g. in an external script. Note that care should be taken not to allow snapshots to grow without bound (though Backrest will typically scale and perform well with hundreds to thousands of snapshots).
 
-- **Hooks** hooks are actions that can be configured in response to backup lifecycle events. See the hooks page of the wiki for more details.
+- **Hooks** hooks are actions that can be configured in response to backup lifecycle events. See the [hooks documentation](/docs/hooks) for more information and the [hooks cookbook](/cookbooks/command-hook-examples) for examples.
 - **Backup Flags** flags that are specific to the backup command (e.g. `--force` to force a full scan of all files rather than relying on metadata). These flags are passed directly to the restic command.
   - `--one-file-system` will prevent restic crossing filesystem boundaries.
   - `--force` will force rereading all files on each backup rather than relying on metadata (e.g. last modified time). This can be much slower.

docs/content/2.docs/1.operations.md

@@ -14,7 +14,7 @@ When running restic commands, Backrest injects the environment variables configu
 
 [Restic docs on backup](https://restic.readthedocs.io/en/latest/040_backup.html)
 
-A backup operation creates a snapshot of your data and sends it to a repository. The snapshot is created using the `restic backup` command.
+Backups are scheduled under plan settings in the UI. A backup operation creates a snapshot of your data and sends it to a repository. The snapshot is created using the `restic backup` command.
 
 As the backup runs Backrest will display the progress of the backup operation in the UI. The progress information is parsed from the JSON output of the `restic backup` command.
 
@@ -23,20 +23,23 @@ The backup flow is as follows
 - Hook trigger: `CONDITION_SNAPSHOT_START`, if any hooks are configured for this event they will run.
   - If any hook exits with a non-zero status, the hook's failure policy will be applied (e.g. cancelling or failing the backup operation).
 - The `restic backup` command is run. The newly craeted snapshot is tagged with the ID of the plan creating it e.g. `plan:{PLAN_ID}`.
-- The summary event is parsed from the backup and is stored in the operation's metadata. This includes: files added, files changed, files unmodified, total files processed, bytes added, bytes processed, and most importantly the snapshot ID.
-- If an error occurred: hook trigger `CONDITION_SNAPSHOT_ERROR`, if any hooks are configured for this event they will run.
-- Hook trigger: `CONDITION_SNAPSHOT_END`, if any hooks are configured for this event they will run. This condition is always triggered even if the backup failed.
+- On backup completion
+  - The summary event is parsed from the backup and is stored in the operation's metadata. This includes: files added, files changed, files unmodified, total files processed, bytes added, bytes processed, and most importantly the snapshot ID.
+  - If an error occurred: hook trigger `CONDITION_SNAPSHOT_ERROR`, if any hooks are configured for this event they will run.
+  - If successful: hook trigger `CONDITION_SNAPSHOT_SUCCESS`, if any hooks are configured for this event they will run. 
+  - Finally `CONDITION_SNAPSHOT_END` is triggered irrespective of success or failure, if any hooks are configured for this event they will run. This condition is always triggered even if the backup failed.
+ - If a retention policy is set (e.g. not `None`) a forget operation is triggered for the backup plan.
 
-If the backup completed successfully, Backrest triggers followup operations
+Created backups are tagged with some Backrest specific metadata:
 
-- If a retention policy is set (e.g. not `None`) a forget operation is triggered for the backup plan.
-- If a prune policy is set (e.g. not `None`) a prune operation is triggered for the backup plan if it has been long enough since the last prune operation.
+ - `plan:{PLAN_ID}` - the ID of the plan that created the snapshot. This is used to group snapshots by plan in the UI.
+ - `created-by:{INSTANCE_ID}` - the unique ID of the Backrest instance that created the snapshot. This is used to group snapshots by instance in the UI. Notably, this not necessarily the same as the hostname tracked by restic in the snapshot.
 
 #### Forget
 
 [Restic docs on forget](https://restic.readthedocs.io/en/latest/060_forget.html)
 
-A forget operation marks old snapshots for deletion but does not remove data from storage until a prune runs. The forget operation is run using the `restic forget --tag plan:{PLAN_ID}` command.
+Forget operations are scheduled by the "forget policy" configured in plan settings in the UI. Forget operations run after backups. A forget operation marks old snapshots for deletion but does not remove data from storage until a prune runs. The forget operation is run using the `restic forget --tag plan:{PLAN_ID}` command.
 
 Retention policies are mapped to forget arguments:
 
@@ -49,11 +52,32 @@ Note that forget will never run for a plan if the forget policy is set to `None`
 
 [Restic docs on prune](https://restic.readthedocs.io/en/latest/060_forget.html)
 
-A prune operation removes data from storage that is no longer referenced by any snapshot. The prune operation is run using the `restic prune` command. Prune operations apply to the entire repo (e.g. are not scoped to data unreferenced by a specific plan).
+Prune operations are scheduled under repo settings in the UI. A prune operation removes data from storage that is no longer referenced by any snapshot. The prune operation is run using the `restic prune` command. Prune operations apply to the entire repo and will show up under the `_system_` plan in the Backrest UI.
 
 Prunes are run in compliance with a prune policy which specifies:
 
-- **Max Frequency Days** - the minimum time in days between prune operations. A prune is skipped following a backup if the last prune was less than this many days ago. This is to prevent excessive pruning which can be slow and is bandwidth intensive as prunes may read and rewrite portions of the repository.
+- **Schedule** - the schedule on which to run prune operations. Available types are:
+  - **Disabled** - prune operations are disabled. This means that the repository will grow indefinitely and will never be pruned. You should periodically run a prune operation manually if you choose this option. 
+  - **Cron** - a cron expression specifying when to run prune operations.
+  - **Max Frequency Days** - the minimum number of days that must pass between prune operations.
+  - **Max Frequency Hours** - (Advanced) the minimum number of hours that must pass between prune operations. This is useful for running prune operations more frequently than once per day, typically this is a bad idea as prune operations are expensive and should be run infrequently.
 - **Max Unused Percent** - the maximum percentage of the repository that may be left unused after a prune operation runs. Prune operations will try to repack blobs in the repository if more than this percentage of their is unused (e.g. formerly held data belonging to forgotten snapshots).
 
-Prune operations do not support json output so the logs of a prune operation are not parsed for progress information, they are instead displayed as raw text in the Backrest UI as they are received from the restic process.
+::alert{type="info"}
+Prune operations are costly and may read a significant portion of your repo. It is recommended to run them infrequently (e.g. monthly or every 30 days).
+::
+
+#### Check
+
+[Restic docs on check](https://restic.readthedocs.io/en/latest/080_check.html)
+
+Check operations are scheduled under repo settings in the UI. A check operation verifies the integrity of the repository. The check operation is run using the `restic check` command. Check operations apply to the entire repo and will show up under the `_system_` plan in the Backrest UI.
+
+Checks are run in compliance with a check policy which specifies:
+
+- **Schedule** - the schedule on which to run check operations. Available types are:
+  - **Disabled** - check operations are disabled. You should periodically run a check operation manually if you choose this option.
+  - **Cron** - a cron expression specifying when to run check operations.
+  - **Max Frequency Days** - the minimum number of days that must pass between check operations.
+  - **Max Frequency Hours** - (Advanced) the minimum number of hours that must pass between check operations. This is useful for running check operations more frequently than once per day, typically this is a bad idea as check operations are expensive and should be run infrequently.
+- **Read Data %** - the percentage of the repository that should be read during a check operation.  A value of 100% will check the entire repository which can be very slow and can be expensive if your provider bills for egress bandwidth. Reading back data is intended to verify the integrity of pack files on-disk for potentially unreliable storage (e.g. an HDD running without parity). It typically does not provide much value for a reliable storage provider and can be set to a low percentage or disabled.

docs/content/2.docs/2.hooks.md

@@ -11,6 +11,7 @@ Available event types are
 - `CONDITION_SNAPSHOT_END` the end of a backup operation (e.g. corresponds to `restic backup` completing). Note that Snapshot End will still be called if a backup failed.
 - `CONDITION_SNAPSHOT_ERROR` an error occurred during a backup operation (e.g. `restic backup` returned a non-zero exit code OR invalid output).
 - `CONDITION_SNAPSHOT_WARNING` a warning occurred during a backup operation (e.g. a file was partially read).
+- `CONDITION_SNAPSHOT_SUCCESS` a backup operation completed successfully.
 - Prune hooks:
   - `CONDITION_PRUNE_START` the start of a prune operation (e.g. corresponds to a call to `restic prune`, supports error behavior)
   - `CONDITION_PRUNE_SUCCESS` the end of a prune operation e.g. `restic prune` completed successfully.