docs(all) Update staging compose and deployment notes

deploy/README.md

@@ -4,18 +4,16 @@ The site can run on a single Linux virtual machine, with a separate database ser
 
 The virtual machine runs several services, coordinated by docker-compose:
 
-- Traefik proxy receives requests from the web, terminates SSL connections and
-  handles basic authentication.
 - Frontend React application, built using node and npm. In production this is
   stored and served as static files (HTML/JS/CSS).
 - Vector tileserver, tileserver-gl-light, depends on node
 - Raster tileserver, terracotta, depends on gunicorn and Python 3.10
 - Backend Python application, depends on uvicorn, fastapi and Python 3.10
 
-The frontend application source code is held in
-[this repository](https://github.com/nismod/infra-risk-vis/) and this guide
-assumes that it is built using node and npm locally on a development machine.
-It would be possible to build directly on the server in a working directory.
+The application source code is held in [this
+repository](https://github.com/nismod/irv-jamaica/) and this guide assumes that
+it is built using node and npm locally on a development machine. It would be
+possible to build directly on the server in a working directory.
 
 To build and deploy the site:
 
@@ -84,18 +82,63 @@ install docker and docker-compose.
 
 ## Basic authentication
 
-Create a password file for HTTP Basic Authentication if it doesn't already exist:
+The J-SRAT app can be configured to use [HTTP Basic
+Authentication](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication)
+to authenticate users. In this case, the connection must always use HTTPS, to
+ensure that credentials are protected.
+
+The user accounts are stored in a text file on the server, which is used by the
+Nginx server, which terminates all connections and reverse-proxies connections
+to the other app services. See the
+[Nginx docs](https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-http-basic-authentication/)
+for more on this configuration.
+
+Create a password file for HTTP Basic Authentication if it doesn't already
+exist:
 
 ```bash
 sudo touch /var/www/auth/.htpasswd
 ```
 
-Add a user to the password file (will prompt for password):
+### Add a user account
+
+Optionally use the command-line password generation utility, `pwgen`, to generate
+passwords. E.g. run the following to generate three 16-character passwords:
+
+    pwgen -N3 16
+
+To add or update a user in the password file (will prompt for password):
 
 ```bash
 sudo htpasswd -B /var/www/auth/.htpasswd new-username
 ```
 
+Test that it worked by visiting the site in a private tab, and entering the new
+username and password when prompted.
+
+### Remove a user account
+
+Edit the file `/etc/nginx/.htpasswd` to remove the line with the relevant username, or run:
+
+    sudo htpasswd -D /etc/nginx/.htpasswd username
+
+Test that deletion worked by visiting the site in a private tab, and entering
+the old username and password when prompted, which should fail to authenticate.
+
+### Certificate renewal
+
+The server can be configured to manage its own SSL certificate, and should
+auto-renew every 90 days, but this may fail. If the certificate is outdated,
+users will see a security warning in the browser when they visit the site.
+
+To renew:
+
+1. Log in to the J-SRAT server via SSH
+2. Stop the NGINX server process: `service nginx stop`
+3. Renew the certificate: `sudo certbot renew`
+4. Start NGINX again: `service nginx start`
+5. Visit the site (hard browser refresh) to check the certificate comes through.
+
 ## Database connection
 
 The `PG*` variables for connection to database, to be replaced with actual details:
@@ -139,10 +182,26 @@ cd /var/www
 docker compose -f docker-compose.prod.yml up -d
 ```
 
+For pulling from the GitHub container registry (GHCR), you will need to follow these
+[instructions for authentication](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry)
+and use a token with `read:packages` scope to read from the GHCR.
+
+The docker compose setup runs frontend, backend, vector and raster tileservers
+and exposes these on high-numbered ports within the machine.
+
+[Nginx](https://nginx.org/en/) is used as a reverse-proxy to terminate incoming
+connections and pass them on to the containerised services.
+
+Find example configuration files in `./etc/nginx/sites-available`. On the
+server, the relevant file should be symlinked to `/etc/nginx/sites-enabled`.
+
+See [certbot docs](https://certbot.eff.org/instructions?ws=nginx&os=ubuntufocal)
+for instructions on setting up or renewing SSL certificates.
+
 ### Publishing docker images
 
 For pushing to the GitHub container registry, you will need to follow these
-[instructions for authentication](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry).
+[instructions for authentication](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry) and use a token with `write:packages` scope.
 
 Build and publish all images:
 

deploy/deploy.sh

@@ -4,25 +4,26 @@ set -x
 
 #
 # Deploy app data
-# - assumes that SSH config is set up to connect to "jsrat1" host
+# - assumes that SSH config is set up to connect to host "jsrat1"
 #
+host="jsrat1"
 BASEDIR=$(dirname $0)
 
 pushd $BASEDIR/..
 
 echo "Running in ${pwd}"
 
 # vector data
-rsync -ravz tileserver/vector/data/ jsrat1:/var/www/tileserver/vector/data
-rsync -ravz tileserver/vector/fonts/ jsrat1:/var/www/tileserver/vector/fonts
-rsync -ravz tileserver/vector/config.json jsrat1:/var/www/tileserver/vector
+rsync -ravz tileserver/vector/data/ $host:/var/www/tileserver/vector/data
+rsync -ravz tileserver/vector/fonts/ $host:/var/www/tileserver/vector/fonts
+rsync -ravz tileserver/vector/config.json $host:/var/www/tileserver/vector
 
 # raster data
-rsync -ravz tileserver/raster/data/ jsrat1:/var/www/tileserver/raster/data
-rsync -ravz tileserver/raster/config.toml jsrat1:/var/www/tileserver/raster
+rsync -ravz tileserver/raster/data/ $host:/var/www/tileserver/raster/data
+rsync -ravz tileserver/raster/config.toml $host:/var/www/tileserver/raster
 
 # docker compose configuration
-rsync -avz docker-compose.prod.yml jsrat1:/var/www/
-rsync -avz envs/prod jsrat1:/var/www/envs/
+rsync -avz docker-compose.prod.yml $host:/var/www/
+rsync -avz envs/prod $host:/var/www/envs/
 
 popd

deploy/deploy_stage.sh

@@ -0,0 +1,29 @@
+#!/usr/bin/env bash
+set -e
+set -x
+
+#
+# Deploy app data
+# - assumes that SSH config is set up to connect to host "jamaica"
+#
+host="jamaica"
+BASEDIR=$(dirname $0)
+
+pushd $BASEDIR/..
+
+echo "Running in ${pwd}"
+
+# # vector data
+# rsync -ravz tileserver/vector/data/ $host:/var/www/tileserver/vector/data
+# rsync -ravz tileserver/vector/fonts/ $host:/var/www/tileserver/vector/fonts
+# rsync -ravz tileserver/vector/config.json $host:/var/www/tileserver/vector
+
+# # raster data
+# rsync -ravz tileserver/raster/data/ $host:/var/www/tileserver/raster/data
+# rsync -ravz tileserver/raster/config.toml $host:/var/www/tileserver/raster
+
+# docker compose configuration
+rsync -avz docker-compose.stage.yml $host:/var/www/
+rsync -avz envs/stage/ $host:/var/www/envs
+
+popd

deploy/etc/nginx/sites-available/jamaica.infrastructureresilience.org

@@ -0,0 +1,170 @@
+##
+# You should look at the following URL's in order to grasp a solid understanding
+# of Nginx configuration files in order to fully unleash the power of Nginx.
+# https://www.nginx.com/resources/wiki/start/
+# https://www.nginx.com/resources/wiki/start/topics/tutorials/config_pitfalls/
+# https://wiki.debian.org/Nginx/DirectoryStructure
+#
+# Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples.
+##
+
+# Route to docker services
+upstream frontend {
+        server 127.0.0.1:3000;
+}
+
+upstream backend {
+        server 127.0.0.1:3001;
+}
+
+upstream vector_tileserver {
+        server 127.0.0.1:3002;
+}
+
+upstream raster_tileserver {
+        server 127.0.0.1:3003;
+}
+
+# Set up rate limit
+limit_req_zone $binary_remote_addr zone=slow:10m rate=50r/s;
+
+server {
+	listen 80 default_server;
+	listen [::]:80 default_server;
+
+	# Note: You should disable gzip for SSL traffic.
+	# See: https://bugs.debian.org/773332
+	#
+	# Read up on ssl_ciphers to ensure a secure configuration.
+	# See: https://bugs.debian.org/765782
+	#
+	# Self signed certs generated by the ssl-cert package
+	# Don't use them in a production server!
+	#
+	# include snippets/snakeoil.conf;
+
+	root /var/www/html;
+	index index.html
+
+	server_name jamaica.infrastructureresilience.org;
+
+	if ($http_origin = ''){
+		set $http_origin "*";
+	}
+
+	location / {
+		# basic authentication - use htpasswd to add users
+		auth_basic "Access restricted";
+		auth_basic_user_file /etc/nginx/.htpasswd;
+
+		# limit rate of requests to root of site
+		limit_req zone=slow burst=50 nodelay;
+
+		# allow CORS with auth
+		add_header Access-Control-Allow-Origin "$http_origin" always;
+		add_header 'Access-Control-Allow-Credentials' 'true' always;
+		add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
+		add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
+
+		# Pass to upstream
+		proxy_pass http://frontend;
+	}
+
+
+	location /static {
+		# basic authentication - use htpasswd to add users
+		auth_basic "Access restricted";
+		auth_basic_user_file /etc/nginx/.htpasswd;
+
+		# Pass to upstream
+		proxy_pass http://frontend;
+		expires 1y;
+		access_log off;
+		add_header Cache-Control "public";
+
+		# allow CORS with auth
+		add_header Access-Control-Allow-Origin "$http_origin" always;
+		add_header 'Access-Control-Allow-Credentials' 'true' always;
+		add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
+		add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
+	}
+
+	location /vector {
+		# basic authentication - use htpasswd to add users
+		auth_basic "Access restricted";
+		auth_basic_user_file /etc/nginx/.htpasswd;
+
+		proxy_pass http://vector_tileserver;
+		rewrite ^/vector/(.+) /$1 break;
+		proxy_set_header Host $host/vector;
+		proxy_set_header X-Real-IP $remote_addr;
+		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+		proxy_set_header X-Forwarded-Proto $scheme;
+
+		# allow CORS with auth
+		add_header Access-Control-Allow-Origin "$http_origin" always;
+		add_header 'Access-Control-Allow-Credentials' 'true' always;
+		add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
+		add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
+	}
+
+	location /raster {
+		# basic authentication - use htpasswd to add users
+		auth_basic "Access restricted";
+		auth_basic_user_file /etc/nginx/.htpasswd;
+
+		proxy_pass http://raster_tileserver;
+		rewrite ^/raster/(.+) /$1 break;
+		proxy_set_header Host $host;
+		proxy_set_header X-Real-IP $remote_addr;
+		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+		proxy_set_header X-Forwarded-Proto $scheme;
+
+		# allow CORS with auth
+		add_header Access-Control-Allow-Origin "$http_origin" always;
+		add_header 'Access-Control-Allow-Credentials' 'true' always;
+		add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
+		add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
+	}
+
+	location /api {
+		# basic authentication - use htpasswd to add users
+ 		auth_basic "Access restricted";
+		auth_basic_user_file /etc/nginx/.htpasswd;
+
+		proxy_pass http://backend;
+		rewrite ^/api/(.+) /$1 break;
+
+		proxy_set_header Host $http_host;
+		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+		proxy_set_header X-Forwarded-Proto $scheme;
+		proxy_redirect off;
+		proxy_buffering off;
+
+		# allow CORS with auth
+		add_header Access-Control-Allow-Origin "$http_origin" always;
+		add_header 'Access-Control-Allow-Credentials' 'true' always;
+		add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
+		add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
+	}
+
+	listen [::]:443 ssl ipv6only=on; # managed by Certbot
+	listen 443 ssl; # managed by Certbot
+	ssl_certificate /etc/letsencrypt/live/jamaica.infrastructureresilience.org/fullchain.pem; # managed by Certbot
+	ssl_certificate_key /etc/letsencrypt/live/jamaica.infrastructureresilience.org/privkey.pem; # managed by Certbot
+	include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
+	ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
+}
+
+# Redirect from HTTP to HTTPS
+server {
+	if ($host = jamaica.infrastructureresilience.org) {
+		return 301 https://$host$request_uri;
+	} # managed by Certbot
+
+
+	listen 80 ;
+	listen [::]:80 ;
+	server_name jamaica.infrastructureresilience.org;
+	return 404; # managed by Certbot
+}

deploy/provision.sh

@@ -6,11 +6,12 @@
 # - assuming this script is run as a user in groups sudo and jsrat_admin
 #
 
-# Install helper apt packages
+# Install helper apt packages and nginx
 sudo apt-get update
 sudo apt-get install -y \
   ca-certificates curl gnupg \
-  apache2-utils
+  apache2-utils \
+  nginx
 
 # Install docker
 sudo install -m 0755 -d /etc/apt/keyrings