init workflow

Initializes Confluence workflow by:

1. Ensuring the EFS directories are set up
2. Downloading required data for SWORD, gauges, and reaches of interest subset file (reaches of interest, gauge data, and sword)
3. Uploading continent-setfinder.json file to JSON S3 bucket

Note only downloads gauge and sword data if it does not exist.

This is a docker container meant to be run as an AWS Batch job after the input module.

installation

Build a Docker image: docker build -t init_workflow .

execution

AWS credentials will need to be passed as environment variables to the container so that init_workflow may access AWS infrastructure to generate JSON files.

# Credentials
export aws_key=XXXXXXXXXXXXXX
export aws_secret=XXXXXXXXXXXXXXXXXXXXXXXXXX

# Docker run command
docker run --rm --name init_workflow -e AWS_ACCESS_KEY_ID=$aws_key -e AWS_SECRET_ACCESS_KEY=$aws_secret -e AWS_DEFAULT_REGION=us-west-2 init_workflow:latest

deployment

There is a script to deploy the Docker container image and Terraform AWS infrastructure found in the deploy directory.

Script to deploy Terraform and Docker image AWS infrastructure

REQUIRES:

• jq (https://jqlang.github.io/jq/)
• docker (https://docs.docker.com/desktop/) > version Docker 1.5
• AWS CLI (https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)
• Terraform (https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli)

Command line arguments:

[1] registry: Registry URI [2] repository: Name of repository to create [3] prefix: Prefix to use for AWS resources associated with environment deploying to [4] s3_state_bucket: Name of the S3 bucket to store Terraform state in (no need for s3:// prefix) [5] profile: Name of profile used to authenticate AWS CLI commands

Example usage: ``./deploy.sh "account-id.dkr.ecr.region.amazonaws.com" "container-image-name" "prefix-for-environment" "s3-state-bucket-name" "confluence-named-profile"`

Note: Run the script from the deploy directory.