Merge pull request #44 from ccdc-opensource/CSD-5935

dwickett-ccdc · web-flow · commit e99781571410 · 2023-05-26T14:49:28.000+01:00
CSD-5935 Update readme and added macrohub yml file
diff --git a/.gitignore b/.gitignore
@@ -7,6 +7,9 @@ docker-compose.*.yml
 !docker-compose.disable-user-access.yml
 !docker-compose.sample-db-config.yml
 !docker-compose.port-configuration.yml
+!docker-compose.macromolecule-hub.yml
+!docker-compose.sample-enable-sso.yml
+
 
 # Other
 **.swp
diff --git a/README.md b/README.md
@@ -1,14 +1,14 @@
 # On-Site WebCSD
 
-On-Site WebCSD is currently only available to CCDC Research Partners and other invited groups.
+On-Site WebCSD is currently only available to CCDC industrial customers and other invited groups.
 
 This readme contains information required for installation and updates of On-Site WebCSD.
 
-Please use the latest version from the release tab https://github.com/ccdc-opensource/on-site-webcsd/releases, using main may not work as it is not an official release. Releases prior to version 1.0.0 are alpha releases which will not contain all the functionality currently provided in On-Site WebCSD.
+Please use the latest non alpha version from the release tab <https://github.com/ccdc-opensource/on-site-webcsd/releases>, using main may not work as it is not an official release. Releases prior to version 1.0.0 or with <version>-alpha are alpha releases which will not contain all the functionality currently provided in On-Site WebCSD.
 
 Terms and conditions can be found in the license.md file.
 
-Additional information including user configuration and troubleshooting can be found in the wiki at https://github.com/ccdc-opensource/on-site-webcsd/wiki.
+Additional information including user configuration and troubleshooting can be found in the wiki at <https://github.com/ccdc-opensource/on-site-webcsd/wiki>.
 
 ## Prerequisites
 
@@ -23,9 +23,12 @@ Click on the release you want to use, and download the source code zip and unpac
 
 ## Initial recommended specification
 
-Recommended requirements:
+Recommended requirements for On-Site WebCSD:
 170GB free hd space, 32GB ram, 8 core cpu.
 
+Recommended requirements for On-Site WebCSD including Macromolecule Hub:
+400GB free hd space, 32GB ram, 8 core cpu.
+
 On-Site WebCSD should work with any linux OS that meets the requirements to run Docker, but official support is provided by CCDC on the following platforms:
 
 - Linux - Intel compatible, 64-bit:
@@ -38,6 +41,11 @@ On-Site WebCSD should work with any linux OS that meets the requirements to run
 
 On-Site WebCSD can be configured to read from in-house databases. To enable these databases, copy and rename the file `docker-compose.sample-db-config.yml` to `docker-compose.db-config.yml` and edit the `volumes` section to point to any in-house databases and edit the `environment` section to configure the application to recognise these databases. More information is given in the notes & example sections of the sample file. This acts as an [override file](https://docs.docker.com/compose/extends/) which you will have to include in the startup command.
 
+## CSD-Theory Database Configuration
+
+If wanting to use WebCSD with CSD-Theory, ensure the steps in [link](#In-house Database Configuration) have been followed.
+There are examples of CSD-Theory configurations in the `docker-compose.sample-db-config.yml` file.
+
 ## Installation
 
 After unpacking the release source code onto the server on which the software will be installed you will need to go into the on-site-webcsd directory and copy the environment file `sample.env` as `.env` then populate with suitable values.
@@ -56,20 +64,33 @@ CSD_DB_PASSWORD=A password of your choosing
 CSD_CACHE_PASSWORD=A password of your choosing
 ```
 
-Where stated, some of these variables will be provided by CCDC; all other variables are for the user to generate and set. Once the variables file has been populated, login to the CCDC container registry and bring up the stack:
+Where stated, some of these variables will be provided by CCDC; all other variables are for the user to generate and set.
+Once the variables file has been populated, login to the CCDC container registry and bring up the stack:
 
 ```
 docker login -u <user> -p <password> ccdcrepository.azurecr.io
 
 # or to be prompted for the password
 docker login -u <user> --password-stdin ccdcrepository.azurecr.io
 
-#Use one of the following two commands depending on if you have in-house database configuration
-#Use this command if you have no in-house database configuration
+# As of v2.0.0, containers run as non root users. Because of this you will need to run the following in the on-site-webcsd directory:
+sudo chown -R ccdc:ccdc userdata/
+
+# You will also need to ensure the user "ccdc" has read access to any in-house or CSP databases by using the command above on relevant directories. 
+
+#Use one of the following commands: 
+
+#Use this command if you have no in-house databases and don't want to use macromolecule hub
 docker compose up -d
 
-#Use this command if you have in-house database configuration
+#Use this command if you have in-house databases and don't want to use macromolecule hub 
 docker compose -f docker-compose.yml -f docker-compose.db-config.yml up -d
+
+#Use this command if you have in-house databases and want macromolecule hub 
+docker compose -f docker-compose.macromolecule-hub.yml -f docker-compose.db-config.yml up -d
+
+#Use this command if you have no in-house databases and want macromolecule hub
+docker compose -f docker-compose.macromolecule-hub.yml up -d
 ```
 
 ## Updates
@@ -83,12 +104,19 @@ Once the latest installation files have been obtained, to update the software, p
 docker compose pull
 
 docker compose down
-#Use one of the following two commands depending on if you have in-house database configuration
-#Use this command if you have no in-house database configuration
+#Use one of the following commands: 
+
+#Use this command if you have no in-house databases and don't want to use macromolecule hub
 docker compose up -d
 
-#Use this command if you have in-house database configuration
+#Use this command if you have in-house databases and don't want to use macromolecule hub 
 docker compose -f docker-compose.yml -f docker-compose.db-config.yml up -d
+
+#Use this command if you have in-house databases and want macromolecule hub 
+docker compose -f docker-compose.macromolecule-hub.yml -f docker-compose.db-config.yml up -d
+
+#Use this command if you have no in-house databases and want macromolecule hub
+docker compose -f docker-compose.macromolecule-hub.yml up -d
 ```
 
 ## Verifying the Installation/Update
@@ -105,8 +133,8 @@ For more information see the [Docker volumes documentation](https://docs.docker.
 
 ## Usage
 
-To access the WebCSD service locally go to http://localhost in a browser.
+To access the WebCSD service locally go to <http://localhost> in a browser.
 
 ## Contact support
 
-If you experience any difficulties with installing or using On-Site WebCSD, please contact our support team at support@ccdc.cam.ac.uk who will be happy to assist you.
+If you experience any difficulties with installing or using On-Site WebCSD, please contact our support team at <support@ccdc.cam.ac.uk> who will be happy to assist you.
diff --git a/docker-compose.macromolecule-hub.yml b/docker-compose.macromolecule-hub.yml
@@ -0,0 +1,220 @@
+version: "3.6"
+
+x-labels: &k8s-labels
+  kompose.image-pull-secret: pullsecret
+
+services:
+  webcsd:
+    environment:
+      - CsdRepository__ServiceLocation=webcsdbackend
+      - CspRepository__ServiceLocation=webcsd-theory
+      - CsdRepository__SketcherFeatures__1=Proteins
+    depends_on:
+      - ccdc-csd-searchservice
+      - csd-request-entry
+    image: ccdcrepository.azurecr.io/onsite/webcsd:2.0.0
+    ports:
+      - 80:80
+    volumes:
+      - ./userdata:/app/OnSite
+    labels:
+      <<: *k8s-labels
+      kompose.service.type: LoadBalancer
+      kompose.service.expose: "true"
+
+  webcsdbackend:
+    depends_on:
+      - ccdc-csd-searchservice
+      - csd-request-entry
+    image: ccdcrepository.azurecr.io/onsite/webcsdbackend:2.0.0
+    environment:
+      - ServiceSettings__StructureInfoLocation=/csd-data/structure-links.csv
+      - CCDC_LICENSING_CONFIGURATION=${CCDC_LICENSING_CONFIGURATION}
+      - ServiceSettings__Databases__1__Name=PDB Database
+      - ServiceSettings__Databases__1__DbeName=pdb
+      - ServiceSettings__Databases__1__ConnectionString=dbe
+    labels:
+      <<: *k8s-labels
+    expose:
+      - 80
+
+  redis:
+    labels:
+      <<: *k8s-labels
+    image: redis
+    user: redis
+    restart: always
+    command: redis-server --requirepass ${CSD_CACHE_PASSWORD}
+    expose:
+      - 6379
+
+  database-server:
+    labels:
+      <<: *k8s-labels
+    image: ccdcrepository.azurecr.io/onsite/macromolecule-hub-database:2.0.0
+    # This can be removed next time db image is updated
+    user: postgres
+    restart: always
+    environment:
+      - POSTGRES_PASSWORD=${CSD_DB_PASSWORD}
+    command:
+      - "postgres"
+      - "-c"
+      - "max_connections=1000"
+    expose:
+      - 5432
+
+  ccdc-csd-substructuresearch:
+    labels:
+      <<: *k8s-labels
+    image: ccdcrepository.azurecr.io/onsite/ccdc-csd-substructuresearch-api:2.0.0
+    depends_on:
+      - database-server
+      - ccdc-csd-resultstore
+      - ccdc-csd-substructure-filter
+      - ccdc-csd-fingerprint
+      - ccdc-csd-deposition
+    environment:
+      - StructureSearchReadConnection=Server=database-server;Port=5432;Database=csd-database;User Id=postgres;Password=${CSD_DB_PASSWORD};SearchPath=substructure
+      - PdbReadConnection=Server=database-server;Port=5432;Database=csd-database;User Id=postgres;Password=${CSD_DB_PASSWORD};SearchPath=structure_optimisation
+      - SubstructureSearchDatabaseConfiguration__Databases__1__Name=pdb
+      - SubstructureSearchDatabaseConfiguration__Databases__1__SettingsKey=PdbReadConnection
+    expose:
+      - 80
+
+  ccdc-csd-deposition:
+    labels:
+      <<: *k8s-labels
+    image: ccdcrepository.azurecr.io/onsite/ccdc-csd-deposition-api:2.0.0
+    depends_on:
+      - database-server
+    environment:
+      - CsdDepositionReadConnection=Server=database-server;Port=5432;Database=csd-database;User Id=postgres;Password=${CSD_DB_PASSWORD};SearchPath=csd_schema
+      - DepositionPdbReadConnection=Server=database-server;Port=5432;Database=csd-database;User Id=postgres;Password=${CSD_DB_PASSWORD};SearchPath=pdb_schema
+      - DatabaseConfiguration__Databases__1__Name=pdb
+      - DatabaseConfiguration__Databases__1__SettingsKey=DepositionPdbReadConnection
+    expose:
+      - 80
+
+  ccdc-csd-textnumericsearch:
+    labels:
+      <<: *k8s-labels
+    depends_on:
+      - database-server
+    image: ccdcrepository.azurecr.io/onsite/ccdc-csd-textnumericsearch-api:2.0.0
+    environment:
+      - CsdReadConnection=Server=database-server;Port=5432;Database=csd-database;User Id=postgres;Password=${CSD_DB_PASSWORD};SearchPath=csd_schema
+      - PdbReadConnection=Server=database-server;Port=5432;Database=csd-database;User Id=postgres;Password=${CSD_DB_PASSWORD};SearchPath=pdb_schema
+      - DatabaseConfiguration__Databases__1__Name=pdb
+      - DatabaseConfiguration__Databases__1__SettingsKey=PdbReadConnection
+    expose:
+      - 80
+
+  ccdc-csd-searchservice:
+    labels:
+      <<: *k8s-labels
+    depends_on:
+      - ccdc-csd-textnumericsearch
+      - ccdc-csd-substructuresearch
+      - ccdc-csd-resultstore
+      - ccdc-csd-similaritysearch
+    image: ccdcrepository.azurecr.io/onsite/ccdc-csd-searchservice-api:2.0.0
+    expose:
+      - 80
+
+  csd-request-entry:
+    labels:
+      <<: *k8s-labels
+    depends_on:
+      - ccdc-csd-deposition
+      - ccdc-csd-crystal-structure-export
+    image: ccdcrepository.azurecr.io/onsite/csd-request-entry-api:2.0.0
+    expose:
+      - 80
+
+  ccdc-csd-formulasearch:
+    labels:
+      <<: *k8s-labels
+    depends_on:
+      - database-server
+    image: ccdcrepository.azurecr.io/onsite/ccdc-csd-formulasearch-api:2.0.0
+    environment:
+      - FormulaSearchReadConnection=Server=database-server;Port=5432;Database=csd-database;User Id=postgres;Password=${CSD_DB_PASSWORD};SearchPath=formula
+    expose:
+      - 80
+
+  ccdc-csd-unitcellsearch:
+    labels:
+      <<: *k8s-labels
+    depends_on:
+      - database-server
+      - ccdc-csd-reducedcell-calculation-service
+    image: ccdcrepository.azurecr.io/onsite/ccdc-csd-unit-cell-search:2.0.0
+    environment:
+      - UnitCellSearchReadConnection=Server=database-server;Port=5432;Database=csd-database;User Id=postgres;Password=${CSD_DB_PASSWORD};SearchPath=unitcell
+      - UnitCellPdbReadConnection=Server=database-server;Port=5432;Database=csd-database;User Id=postgres;Password=${CSD_DB_PASSWORD};SearchPath=pdb_cell_optimisation
+      - DatabaseConfiguration__Databases__1__Name=pdb
+      - DatabaseConfiguration__Databases__1__SettingsKey=UnitCellPdbReadConnection
+    expose:
+      - 80
+
+  ccdc-csd-resultstore:
+    labels:
+      <<: *k8s-labels
+    image: ccdcrepository.azurecr.io/onsite/ccdc-csd-resultstore-api:2.0.0
+    environment:
+      - OrderedCachePassword=${CSD_CACHE_PASSWORD}
+    expose:
+      - 80
+
+  ccdc-csd-similaritysearch:
+    depends_on:
+      - database-server
+      - ccdc-csd-fingerprint
+    image: ccdcrepository.azurecr.io/onsite/ccdc-csd-structure-similarity-search:2.0.0
+    environment:
+      - StructureSimilaritySearchReadConnection=Server=database-server;Port=5432;Database=csd-database;User Id=postgres;Password=${CSD_DB_PASSWORD};SearchPath=similarity
+
+  ccdc-csd-fingerprint:
+    labels:
+      <<: *k8s-labels
+    image: ccdcrepository.azurecr.io/onsite/ccdc-csd-fingerprint-service:2.0.0
+    expose:
+      - 80
+    restart: unless-stopped
+
+  ccdc-csd-substructure-filter:
+    labels:
+      <<: *k8s-labels
+    image: ccdcrepository.azurecr.io/onsite/ccdc-csd-substructure-filter-service:2.0.0
+    expose:
+      - 80
+    restart: unless-stopped
+
+  ccdc-csd-crystal-structure-export:
+    labels:
+      <<: *k8s-labels
+    image: ccdcrepository.azurecr.io/onsite/ccdc-csd-crystal-structure-export-service:2.0.0
+    expose:
+      - 80
+    restart: unless-stopped
+
+  ccdc-csd-reducedcell-calculation-service:
+    labels:
+      <<: *k8s-labels
+    image: ccdcrepository.azurecr.io/onsite/ccdc-csd-reducedcell-calculation-service:2.0.0
+    expose:
+      - 80
+    restart: unless-stopped
+
+  webcsd-theory:
+    labels:
+      <<: *k8s-labels
+    image: ccdcrepository.azurecr.io/onsite/csd-theory:2.0.0
+    environment:
+      - CCDC_LICENSING_CONFIGURATION=${CCDC_LICENSING_CONFIGURATION}
+    expose:
+      - 80
+
+volumes:
+  csd_data: {}
diff --git a/docker-compose.sample-db-config.yml b/docker-compose.sample-db-config.yml
@@ -9,17 +9,17 @@ services:
       - /path/to/database-1:/csd-data/database-1
       - /path/to/database-2:/csd-data/database-2
     environment:
-      - ServiceSettings__Databases__1__Name=database-1
-      - ServiceSettings__Databases__1__ConnectionString=/csd-data/database-1
-      - ServiceSettings__Databases__2__Name=database-2
-      - ServiceSettings__Databases__2__ConnectionString=/csd-data/database-2
+      - ServiceSettings__Databases__2__Name=database-1
+      - ServiceSettings__Databases__2__ConnectionString=/csd-data/database-1
+      - ServiceSettings__Databases__3__Name=database-2
+      - ServiceSettings__Databases__3__ConnectionString=/csd-data/database-2
   # Notes:
   #   The path in which the databases must be mounted is `/csd-data`.
   #   For each database there must be 2 corresponding environment variables:
   #      - ServiceSettings__Databases__N__Name=<database name>
   #      - ServiceSettings__Databases__N__ConnectionString=<database path>
   #     where
-  #      - N is a unique integer for each pair of variables
+  #      - N is a unique integer for each database, which should start from 2
   #      - <database name> is how the name of the database will appear in the UI
   #      - <database path> should match the right side of the volume definition above.
   #
@@ -87,7 +87,6 @@ services:
   #     volumes:
   #       - /path/to/CSD-Theory Landscape Database:/app/databases
   # This will create a persistent volume and claim for the location of the landscape database.
-  # Please refer to the README regarding additional steps required for implementing CSD Theory.
   #########################################################################################################
 
   # IMPORTANT: The CSD Search database must also be included in the webcsdbackend volumes.
@@ -106,10 +105,12 @@ services:
   # to use these databases the csd-theory section of the file will look like this
   #
   # services:
-  #   csd-theory:
+  #   webcsd-theory:
   #     volumes:
   #       - /usr/local/data/metadata-db.db:/app/CSPDatabase.db
   #       - /usr/local/data/landscape-db.csdsqlx:/app/databases/landscape-db.csdsqlx
+  #     environment:
+  #       - ReportSettings__CspStructureDatabaseConnectionString=/app/databases/landscape-db.csdsqlx
   #
   # Also add an entry to webcsdbackend for the landscape database
   #
diff --git a/docker-compose.sample-enable-sso.yml b/docker-compose.sample-enable-sso.yml
diff --git a/docker-compose.yml b/docker-compose.yml
