Merge commit 'dbf4751425bf5a85de0664b367d898936cfd2e54'

Author: robvanmieghem

.github/workflows/README.md

@@ -2,7 +2,12 @@
 This page contains some documentation on the workflows for this repository.
 
 ## build_test
-The workflow build_test is used to build and test the code (see build_test.yaml). We are using a custom docker image for building and testing the code. You can find the image on our [Docker Hub](https://hub.docker.com/repository/docker/threefolddev/tfchain). The dockerfile build_test.Dockerfile was used to build that image. If the image no longer meets the expectations please follow these steps:
+The workflow build_test is used to build and test the code (see build_test.yaml). Notice that the binaries are being cached so that the build process is sped up. Once the binaries are build the pipeline will run both the unit tests and the integration tests. This can take up to 30 minutes. The pipeline is ran on every commit to a PR and also when the PR has been merged with development. PRs should only be merged if the pipeline was green (if all tests passed).
+
+For performance reasons we are using a self hosted runner for running the pipeline. The runner will only run one pipeline at a time which means that all other runs will be queued. As the pipeline is ran on every commit it will thus also queue runs of consecutive pushed commits. We strongly advice to add `[skip ci]` to the commit messages whenever possible (when the run of a pipeline can be skipped). A pipeline can also be canceled [here](https://github.com/threefoldtech/tfchain/actions). 
+
+### Docker image
+We are using a custom docker image for building and testing the code. You can find the image on our [Docker Hub](https://hub.docker.com/repository/docker/threefolddev/tfchain). The dockerfile build_test.Dockerfile was used to build that image. If the image no longer meets the expectations please follow these steps:
 
 1) Update the dockerfile as required (add what you need)
 2) Build the new image (execute the comment with .github/workflows as working directory and make sure to increment the version):

.github/workflows/build_test.Dockerfile

@@ -1,8 +1,7 @@
 FROM ubuntu:20.04
 ENV DEBIAN_FRONTEND=noninteractive
-COPY clean_disk_space.sh clean_disk_space.sh
-RUN apt-get update && \
-    apt-get install -y \
+RUN apt update && \
+    apt install -y \
         build-essential \
         clang \
         cmake \
@@ -12,16 +11,20 @@ RUN apt-get update && \
         libclang-dev \
         lld \
         lldb \
-        python3 \
-        python3-pip \
+        software-properties-common \
         tar \
         zstd && \
+    add-apt-repository ppa:deadsnakes/ppa && \
+    apt install -y python3.10 && \
+    curl https://bootstrap.pypa.io/get-pip.py > get-pip.py && \
+    python3.10 get-pip.py && \
+    rm -rf get-pip.py && \
     curl https://sh.rustup.rs -sSf | sh -s -- -y && \
     $HOME/.cargo/bin/rustup install nightly-2022-05-11 && \
     # cleanup image 
     rm -rf /var/lib/apt/lists/* && \
-    apt-get clean && \
-    apt-get autoclean && \
-    apt-get autoremove && \
+    apt -y clean && \
+    apt -y autoclean && \
+    apt -y autoremove && \
     rm -rf /tmp/* 
 RUN /bin/bash

.github/workflows/build_test.yaml

@@ -8,14 +8,16 @@ jobs:
   build-and-test:
     runs-on: [self-hosted, poc]
     container:
-      image: threefolddev/tfchain:0
+      image: threefolddev/tfchain:1
       env:
         DEBIAN_FRONTEND: noninteractive
     steps:
       - uses: actions/checkout@v3
 
       - name: Cache build
         uses: actions/cache@v3
+        timeout-minutes: 6
+        continue-on-error: true
         with:
           path: |
             ~/.cargo/bin/
@@ -26,7 +28,6 @@ jobs:
           key: ${{ runner.os }}-tfchain-cargo-${{ hashFiles('**/Cargo.lock') }}
           restore-keys: ${{ runner.os }}-tfchain-cargo-
 
- 
       - name: Build
         run: |
           cd substrate-node
@@ -39,3 +40,10 @@ jobs:
           $HOME/.cargo/bin/cargo +nightly-2022-05-11 test --no-fail-fast
           cd pallets
           $HOME/.cargo/bin/cargo +nightly-2022-05-11 test --no-fail-fast
+
+      - name: Integration tests
+        run: |
+          python3.10 -m pip install robotframework cryptography substrate-interface
+          cd substrate-node/tests
+          robot -d _output_tests/ .
+

.gitpod.Dockerfile

@@ -0,0 +1,19 @@
+# 1. Record architecture decisions
+
+Date: 2022-10-12
+
+## Status
+
+Accepted
+
+## Context
+
+We need to record the architectural decisions made on this project.
+
+## Decision
+
+We will use Architecture Decision Records, as [described by Michael Nygard](http://thinkrelevance.com/blog/2011/11/15/documenting-architecture-decisions).
+
+## Consequences
+
+See Michael Nygard's article, linked above. For a lightweight ADR toolset, see Nat Pryce's [adr-tools](https://github.com/npryce/adr-tools).