Merge branch 'eero/reorg-docs' into 'master'

[NODE-1356] Update relocated rootfs references

This updates most references to remove full paths. Repo paths are now decoupled from their destinations, this way we won't have to keep updating comments. 

See merge request dfinity-lab/public/ic!18850

.gitlab/CODEOWNERS

@@ -52,14 +52,14 @@ go_deps.bzl               @dfinity-lab/teams/idx
 # [IC-OS]
 /ic-os/                                          @dfinity-lab/teams/node-team
 /ic-os/boundary-guestos/                         @dfinity-lab/teams/boundarynode-team @dfinity-lab/teams/node-team
-/ic-os/guestos/tests/e2e-benchmarking-suite.py   @dfinity-lab/teams/runtime-owners
+/ic-os/rootfs/boundary-guestos/                  @dfinity-lab/teams/boundarynode-team @dfinity-lab/teams/node-team
 /toolchains/                                     @dfinity-lab/teams/node-team
 
 # [metrics-proxy]
-/ic-os/hostos/rootfs/etc/metrics-proxy.yaml                             @dfinity-lab/teams/DRE
-/ic-os/hostos/rootfs/etc/systemd/system/metrics-proxy.service           @dfinity-lab/teams/DRE
-/ic-os/guestos/rootfs/etc/metrics-proxy.yaml                             @dfinity-lab/teams/DRE
-/ic-os/guestos/rootfs/etc/systemd/system/metrics-proxy.service           @dfinity-lab/teams/DRE
+/ic-os/rootfs/hostos/etc/metrics-proxy.yaml                              @dfinity-lab/teams/DRE
+/ic-os/rootfs/hostos/etc/systemd/system/metrics-proxy.service            @dfinity-lab/teams/DRE
+/ic-os/rootfs/guestos/etc/metrics-proxy.yaml                             @dfinity-lab/teams/DRE
+/ic-os/rootfs/guestos/etc/systemd/system/metrics-proxy.service           @dfinity-lab/teams/DRE
 
 # [GitLab-CI]
 /.pre-commit-config.yaml                                                @dfinity-lab/teams/idx

ic-os/boundary-guestos/context/Dockerfile

@@ -35,7 +35,7 @@ USER root:root
 
 # Bake the nns public key into the image
 # For the dev image the public key will come from the usb stick
-# See get_nns_pem_path in ic-os/boundary-guestos/rootfs/opt/ic/bin/helpers.shlib
+# See get_nns_pem_path in helpers.shlib
 RUN printf '%s\n' >/etc/nns_public_key.pem \
     '-----BEGIN PUBLIC KEY-----' \
     'MIGCMB0GDSsGAQQBgtx8BQMBAgEGDCsGAQQBgtx8BQMCAQNhAIFMDm7HH6tYOwi9' \

ic-os/docs/Configuration.adoc

@@ -13,18 +13,18 @@ In production, configuration is propagated from a partition on the USB installer
 
 === SetupOS -> HostOS
 
-SetupOS validates, sanitizes, and copies all of its configuration files to the HostOS config partition. Refer to link:../setupos/rootfs/opt/ic/bin/config.sh[config.sh] link:../setupos/rootfs/opt/ic/bin/devices.sh[devices.sh]
+SetupOS validates, sanitizes, and copies all of its configuration files to the HostOS config partition. Refer to link:../rootfs/setupos/opt/ic/bin/config.sh[config.sh] link:../rootfs/setupos/opt/ic/bin/devices.sh[devices.sh]
 
 === HostOS -> GuestOS
 
 HostOS builds the "bootstrap config image". Refer to link:../scripts/build-bootstrap-config-image.sh[build-bootstrap-config-image.sh]
 
 The bootstrap config image contains a tar file with the GuestOS configuration files.
-When the HostOS launches the GuestOS, the bootstrap config image is attached to the GuestOS as a virtual USB. Refer to link:../hostos/rootfs/opt/ic/share/guestos.xml.template[guestos.xml.template]
+When the HostOS launches the GuestOS, the bootstrap config image is attached to the GuestOS as a virtual USB. Refer to link:../rootfs/hostos/opt/ic/share/guestos.xml.template[guestos.xml.template]
 
 When the GuestOS boots, it checks for available removable media devices (i.e. the bootstrap config image). If such a device is found, the media must contain a VFAT filesystem and a single file called `ic-bootstrap.tar`.
 
-GuestOS then reads from `ic-bootstrap.tar` (link:../guestos/rootfs/opt/ic/bin/bootstrap-ic-node.sh[see bootstrap-ic-node.sh]), and the contents are unpacked and distributed in one of two places:
+GuestOS then reads from `ic-bootstrap.tar` (link:../rootfs/guestos/opt/ic/bin/bootstrap-ic-node.sh[see bootstrap-ic-node.sh]), and the contents are unpacked and distributed in one of two places:
 
 * *System state:* Node initialization material (initial registry & keys) is copied directly into target directories for replica use.
 * *Config partition:* All other contents of the injected config are copied into the `/boot/config` directory. These configurations need to be available on subsequent boots (not just the first boot of the system), and need to be preserved across upgrades.

ic-os/docs/Network-Configuration.adoc

@@ -7,7 +7,7 @@ Network configuration details for each IC-OS:
 * SetupOS
 ** Basic network connectivity is checked via pinging nns.ic0.app and the default gateway. Virtually no network traffic goes through SetupOS.
 * HostOS
-** The br6 bridge network interface is set up and passed to the GuestOS VM through qemu (refer to hostos/rootfs/opt/ic/share/guestos.xml.template).
+** The br6 bridge network interface is set up and passed to the GuestOS VM through qemu (refer to guestos.xml.template).
 * GuestOS
 ** An internet connection is received via the br6 bridge interface from qemu.
 

ic-os/docs/Services.adoc

@@ -7,12 +7,12 @@ To see the full list of packages included in each IC-OS, refer to the context/pa
 
 == Services
 
-In addition to the regular, built-in Ubuntu services, a unique set of systemd services are added or managed for each IC-OS. Some services are enabled in context/Dockerfile, and custom services are defined at rootfs/etc/systemd/services.
+In addition to the regular, built-in Ubuntu services, a unique set of systemd services are added or managed for each IC-OS. Some services are enabled in context/Dockerfile, and custom services are defined at etc/systemd/services.
 
 The specific systemd services for each IC-OS are as follows:
 
 [NOTE]
-These lists may be out-of-date. For the source of truth, see each OSes `context/Dockerfile` and `rootfs/etc/systemd`.
+These lists may be out-of-date. For the source of truth, see each OSes `context/Dockerfile` and `etc/systemd`.
 
 === SetupOS
 

ic-os/docs/Upgrades.adoc

@@ -39,11 +39,11 @@ Note that the upgrade images include only the boot and root partitions in a tar
 
 Each HostOS/GuestOS rootfs folder contains the manageboot script, used to install the upgrade image:
 
-    rootfs/opt/ic/bin/manageboot.sh upgrade-install update-img.tar
-    rootfs/opt/ic/bin/manageboot.sh upgrade-commit
+    manageboot.sh upgrade-install update-img.tar
+    manageboot.sh upgrade-commit
 
 Following the execution of these commands, the system will reboot into the newly installed version. Note that unless confirmation of full operational functionality is provided, the system will rollback to the previous HostOS/GuestOS on the subsequent reboot:
 
-    rootfs/opt/ic/bin/manageboot.sh confirm
+    manageboot.sh confirm
 
-For more information on the upgrade commands, see https://github.com/dfinity/ic/blob/master/ic-os/guestos/rootfs/opt/ic/bin/manageboot.sh[rootfs/opt/ic/bin/manageboot.sh]
+For more information on the upgrade commands, see https://github.com/dfinity/ic/blob/master/ic-os/rootfs/guestos/opt/ic/bin/manageboot.sh[rootfs/opt/ic/bin/manageboot.sh]

ic-os/guestos/docs/Interface.adoc

@@ -8,5 +8,5 @@ This document covers the GuestOS interface requirements, providing all the infor
 ** Details on populating the GuestOS config partition
 * link:../../../rs/ic_os/vsock/README.md[Vsock]
 ** Communication between GuestOS and HostOS
-* link:../../hostos/rootfs/opt/ic/share/guestos.xml.template[guestos.xml.template]
+* link:../../rootfs/hostos/opt/ic/share/guestos.xml.template[guestos.xml.template]
 ** Details on resource allocations used in production

ic-os/guestos/docs/SELinux-Build.adoc

@@ -11,7 +11,7 @@ the kernel is booted using correct boot arguments
 (+security=selinux+ +selinux=1+).
 
 Additional policy modules are built using the Makefile supplied
-by the SELinux distribution (see +ic/ic-os/guestos/rootfs/prep/prep.sh+).
+by the SELinux distribution (see +prep/prep.sh+).
 The linked policy is produced by inserting the modules into
 the policy state that exists after installation of the
 baseline packages. Crucially, this step as implemented currently
@@ -21,7 +21,7 @@ container whose contents will become the target system.
 
 There are some filesystem / directory entry ordering issues
 affecting build determinism using the standard module build
-setup. See +ic/ic-os/guestos/rootfs/prep/prep.sh+ how this
+setup. See +prep/prep.sh+ how this
 issue is being addressed.
 
 The result of this process is two-fold:

ic-os/guestos/docs/SELinux-Policy.adoc

@@ -33,7 +33,7 @@ detail in the xref:ic-policy-structure[IC security policy structure] section.
 == Policy modules
 
 This section gives an overview of the organisation of
-security policy modules (to be found in +ic/ic-os/guestos/rootfs/prep+).
+security policy modules (to be found in +prep+).
 
 === +systemd-fixes+ and +misc-fixes+
 

ic-os/hostos/README.adoc

@@ -56,9 +56,7 @@ The root partition is formatted as an ext4 file system and is mounted as read-on
   # <file system> <mount point>   <type>  <options>              <dump>  <pass>
   /dev/rootfs     /               ext4    ro,errors=remount-ro   0       1
 
-For details, please refer to the +fstab+ file in:
-
-`ic/ic-os/hostos/rootfs/etc/fstab`
+For details, please refer to the HostOS +fstab+ file.
 
 == System Users
 
@@ -132,9 +130,7 @@ This configuration ensures that the physical CPU topology is reflected in the vi
 
 The hard-coded firewall ruleset is rather restrictive. A new disk-image has to be proposed and blessed in order to update the rules.
 
-Please find the raw NFTables ruleset in:
-
-  ic/ic-os/hostos/rootfs/etc/nftables.conf
+Please find the raw HostOS NFTables ruleset in `nftables.conf`
 
 === Filter
 

ic-os/rootfs/guestos/dev-certs/README.md

@@ -3,7 +3,7 @@ This README file documents how to (re-)generate DFINITY DEV environment root CA
 well as where to apply these files and when to apply these files to make use of them.
 
 ## How to (re-)generate new root CA and signing key
-`cd ic-os/guestos/rootfs/dev-certs`
+`cd dev-certs`
 `sh root_cert_gen.sh`
 
 ## How to apply root CA and signing key to VMs
@@ -13,7 +13,7 @@ Copy `minica.pem` to `/usr/local/share/ca-certificates/` folder of Linux VM, and
 to the `ca-certificates.crt` bundle file sitting under `/etc/ssl/certs` folder, where
 processes running on the VM will be able to treat `/etc/ssl/certs` directory as trusted CAs.
 
-IC-OS reads `/ic-os/guestos/rootfs/dev-certs/` to update its `/etc/ssl/certs/ca-certificates.crt`
+IC-OS reads `dev-certs/` to update its `/etc/ssl/certs/ca-certificates.crt`
 bundle in: `/ic-os/guestos/context/Dockerfile`
 
 ### VM as HTTPS server

ic-os/rootfs/guestos/opt/ic/share/ic.json5.template

@@ -66,7 +66,7 @@
         crypto_root: "/var/lib/ic/crypto",
         // The logic and metrics sockets for communicating with the crypto csp vault.
         // Shall match the values configured in
-        // ic-os/guestos/rootfs/etc/systemd/system/ic-crypto-csp.socket
+        // ic-crypto-csp.socket
         csp_vault_type: { unix_socket: { logic: "/run/ic-node/crypto-csp/socket", metrics: "/run/ic-node/crypto-csp/metrics" } },
     },
 
@@ -119,7 +119,7 @@
     adapters_config: {
         // IPC socket and metrics path for BTC Testnet adapter. This UDS path has to be the same as
         // specified in the systemd socket file.
-        // The BTC adapter socket file is: /ic-os/guestos/rootfs/etc/systemd/system/ic-btc-testnet-adapter.socket
+        // The BTC adapter socket file is: ic-btc-testnet-adapter.socket
         bitcoin_testnet_uds_path: "/run/ic-node/bitcoin-testnet-adapter/socket",
         bitcoin_testnet_uds_metrics_path: "/run/ic-node/bitcoin-testnet-adapter/metrics",
         // IPC socket and metrics path for BTC Mainnet adapter. This UDS path has to be the same as
@@ -128,7 +128,7 @@
         bitcoin_mainnet_uds_metrics_path: "/run/ic-node/bitcoin-mainnet-adapter/metrics",
         // IPC socket and metrics path for canister http adapter. These UDS path has to be the same as
         // specified in the systemd socket file.
-        // The canister http adapter socket file is: /ic-os/guestos/rootfs/etc/systemd/system/ic-https-outcalls-adapter.socket
+        // The canister http adapter socket file is: ic-https-outcalls-adapter.socket
         https_outcalls_uds_path: "/run/ic-node/https-outcalls-adapter/socket",
         https_outcalls_uds_metrics_path: "/run/ic-node/https-outcalls-adapter/metrics",
     },

ic-os/rootfs/hostos/etc/metrics-proxy.yaml

@@ -84,8 +84,7 @@ proxies:
       url: https://guestos:42372/metrics/guestos_replica
       tolerate_bad_tls: true
     # The GuestOS metrics proxy already does filtering.
-    # See ic-os/guestos/rootfs/etc/metrics-proxy.yaml
-    # for the filters.
+    # See the GuestOS metrics-proxy.yaml for the filters.
     label_filters: []
     # The expectation is that clients will scrape at a
     # maximum rate of 1 per 10 seconds (6 / min).
@@ -101,8 +100,7 @@ proxies:
       url: https://guestos:42372/metrics/guestos_node_exporter
       tolerate_bad_tls: true
     # The GuestOS metrics proxy already does filtering.
-    # See ic-os/guestos/rootfs/etc/metrics-proxy.yaml
-    # for the filters.
+    # See the GuestOS metrics-proxy.yaml for the filters.
     label_filters: []
     # The expectation is that clients will scrape at a
     # maximum rate of 1 per 10 seconds (6 / min).

ic-os/rootfs/hostos/etc/nftables.conf

@@ -145,9 +145,7 @@ table ip6 filter {
 
     # [Ref A]
     # ic/testnet/tests/pipeline/pipeline.yml
-    # ic/ic-os/guestos/rootfs/opt/ic/share/ic.json5.template
-    # ic/ic-os/guestos/tests/vmtools.py
-    # ic/ic-os/guestos/tests/Readme.md
+    # ic.json5.template
     # This is used by the qemu-system instances spawned inside of the docker gitlab-runner to allow multiple deterministic dynamic on-the-fly VM "test" nodes for running automated tests. Each docker namespace has its own IP network stack so many of these can be running in parallel at the same time between different runs and they will not interfere with each other as a result.
     # Why "Ref A"? nftables config syntax disallows newlines and comments between the last line of a set and the closing brace.
     auto-merge     # Prevent "Conflicting Intervals" errors

ic-os/setupos/README.adoc

@@ -24,9 +24,9 @@ To build a SetupOS image, refer to the link:../README.adoc[IC-OS README]
 
 The SetupOS installation is initiated by the systemd service unit file `setupos.service`. This service is of type idle, which means the installation is triggered only after every other unit has either completed or started.
 
-The installation process consists of multiple Shell and Python scripts, which can be found in the following directory:
+The installation process consists of multiple Shell and Python scripts, which are placed in the following directory:
 
-  ic-os/setupos/rootfs/opt/ic/bin
+  `/opt/ic/bin`
 
 The sequence of the scripts is defined in the main installation script, `setupos.sh`. The order of execution is as follows:
 

rs/async_utils/src/unix.rs

@@ -2,7 +2,7 @@
 /// socket (UDS) between two processes managed by systemd.
 ///
 /// Existing socket systemd configurations can be found
-/// ic-os/guestos/rootfs/etc/systemd/system/*.socket.
+/// in *.socket files.
 use async_stream::__private::AsyncStream;
 use futures::TryFutureExt;
 use std::{

rs/bitcoin/adapter/src/main.rs

@@ -36,8 +36,8 @@ pub async fn main() {
     // metrics socket as FD(4).
     // SAFETY: The process is managed by systemd and is configured to start with at metrics socket.
     // Additionally this function is only called once here.
-    // Systemd Socket config: ic-os/guestos/rootfs/etc/systemd/system/ic-https-outcalls-adapter.socket
-    // Systemd Service config: ic-os/guestos/rootfs/etc/systemd/system/ic-https-outcalls-adapter.service
+    // Systemd Socket config: ic-https-outcalls-adapter.socket
+    // Systemd Service config: ic-https-outcalls-adapter.service
     if config.incoming_source == IncomingSource::Systemd {
         let stream = unsafe { incoming_from_nth_systemd_socket(2) };
         start_metrics_grpc(metrics_registry.clone(), logger.clone(), stream);

rs/bitcoin/adapter/src/rpc_server.rs

@@ -150,8 +150,8 @@ pub fn start_grpc_server(
                     .add_service(BtcServiceServer::new(btc_adapter_impl))
                     // SAFETY: The process is managed by systemd and is configured to start with at least one socket.
                     // Additionally this function is only called once here.
-                    // Systemd Socket config: ic-os/guestos/rootfs/etc/systemd/system/ic-btc-<testnet,mainnet>-adapter.socket
-                    // Systemd Service config: ic-os/guestos/rootfs/etc/systemd/system/ic-btc-<testnet,mainnet>-adapter.service
+                    // Systemd Socket config: ic-btc-<testnet,mainnet>-adapter.socket
+                    // Systemd Service config: ic-btc-<testnet,mainnet>-adapter.service
                     .serve_with_incoming(unsafe { incoming_from_first_systemd_socket() })
                     .await
                     .expect("gRPC server crashed");

rs/boundary_node/certificate_issuance/README.md

@@ -97,7 +97,7 @@ configuration to include all registered custom domains.
 ## Directory Organization
 
 This directory contains all the code related to custom domains with the exception
-of the [`nginx` configuration](../../../ic-os/boundary-guestos/rootfs/etc/nginx/):
+of the [`nginx` configuration](../../../ic-os/rootfs/boundary-guestos/etc/nginx/):
 
 - [`certificate_issuer`](certificate_issuer/) contains the boundary node service, which handles all registration requests, processes registration tasks and interfaces with the `certificate-orchestrator`.
 - [`certificate_syncer`](certificate_syncer/) contains the boundary node service, which ensures that `nginx` has access to all certificates and keys, and is configured to serve the custom domains.
@@ -303,15 +303,15 @@ When you deploy a boundary node with these configurations, the two services
 automatically start.
 
 The `certificate-issuer` relies on the following services and setup scripts:
-* [setup-certificate-issuer.service](../../../ic-os/boundary-guestos/rootfs/etc/systemd/system/setup-certificate-issuer.service)
-* [setup-certificate-issuer.sh](../../../ic-os/boundary-guestos/rootfs/opt/ic/bin/setup-certificate-issuer.sh)
-* [certificate-issuer.service](../../../ic-os/boundary-guestos/rootfs/etc/systemd/system/certificate-issuer.service)
+* [setup-certificate-issuer.service](../../../ic-os/rootfs/boundary-guestos/etc/systemd/system/setup-certificate-issuer.service)
+* [setup-certificate-issuer.sh](../../../ic-os/rootfs/boundary-guestos/opt/ic/bin/setup-certificate-issuer.sh)
+* [certificate-issuer.service](../../../ic-os/rootfs/boundary-guestos/etc/systemd/system/certificate-issuer.service)
 
 The `certificate-syncer` relies on the following services and setup scripts:
-* [setup-certificate-syncer.service](../../../ic-os/boundary-guestos/rootfs/etc/systemd/system/setup-certificate-syncer.service)
-* [setup-certificate-syncer.sh](../../../ic-os/boundary-guestos/rootfs/opt/ic/bin/setup-certificate-syncer.sh)
-* [certificate-syncer.service](../../../ic-os/boundary-guestos/rootfs/etc/systemd/system/certificate-syncer.service)
+* [setup-certificate-syncer.service](../../../ic-os/rootfs/boundary-guestos/etc/systemd/system/setup-certificate-syncer.service)
+* [setup-certificate-syncer.sh](../../../ic-os/rootfs/boundary-guestos/opt/ic/bin/setup-certificate-syncer.sh)
+* [certificate-syncer.service](../../../ic-os/rootfs/boundary-guestos/etc/systemd/system/certificate-syncer.service)
 
-In addition, the `certificate-syncer` needs a [configuration template](/ic-os/boundary-guestos/rootfs/etc/certificate-syncer/domain.tmpl) to dynamically include all
+In addition, the `certificate-syncer` needs a [configuration template](/ic-os/rootfs/boundary-guestos/etc/certificate-syncer/domain.tmpl) to dynamically include all
 custom domains in the `nginx` configuration. It is part of the boundary node
 root filesystem and is maintained along with this codebase.

rs/boundary_node/certificate_issuance/certificate_issuer/README.md

@@ -28,9 +28,9 @@ certificate authority.
 
 The following three files are used to setup and start the service on the boundary node:
 
-* [setup-certificate-issuer.service](../../../ic-os/boundary-guestos/rootfs/etc/systemd/system/setup-certificate-issuer.service)
-* [setup-certificate-issuer.sh](../../../ic-os/boundary-guestos/rootfs/opt/ic/bin/setup-certificate-issuer.sh)
-* [certificate-issuer.service](../../../ic-os/boundary-guestos/rootfs/etc/systemd/system/certificate-issuer.service)
+* [setup-certificate-issuer.service](../../../ic-os/rootfs/boundary-guestos/etc/systemd/system/setup-certificate-issuer.service)
+* [setup-certificate-issuer.sh](../../../ic-os/rootfs/boundary-guestos/opt/ic/bin/setup-certificate-issuer.sh)
+* [certificate-issuer.service](../../../ic-os/rootfs/boundary-guestos/etc/systemd/system/certificate-issuer.service)
 
 The `/registration` endpoint needs to be publicly accessible. To this end, the
 `nginx` configuration needs to expose and endpoint, which is routed to the certificate issuer.