Warm Tier Repository Copy

[tellistone]

The Problem

Setting up the warm tier is done via this button

it brings up this modal:

There is not a lot of explanation here, and this process has a few tricky steps, so we should both summarise the steps and provide a link to the doc to maximize clarity.

The Solution

The "Create new warm tier repository modal" would be improved by some explanatory copy:

The copy should vary depending on detected search backend. For Opensearch:

Repository locations will not be shown until they are configured, which is performed outside of the Graylog UI.

The process for setting up a Warm Tier Repository is documented fully here.

In summary:

• If planning to use AWS S3 or Google Cloud Storage, first install the corresponding Opensearch Plugin, on all search nodes.
• For a AWS S3 data tiering backend, add your AWS access and secret keys to the OpenSearch keystore on all search nodes - see Opensearch docs on this step here.
• For a Google Cloud Storage Data Tiering backend, add the Service Account Key JSON to the OpenSearch keystore on all search nodes - see Opensearch docs on this step here.
• Make documented changes to the Opensearch configuration file on all search nodes.
• Restart all search nodes.
• In Graylog, return to this modal; select the appropriate location from the drop-down.

For Data Node show the steps as follows:

Repository locations will not be shown until they are configured, which is performed outside of the Graylog UI.

The process for setting up a Warm Tier Repository is documented fully here.

In summary:

• If using Google Cloud Storage, on all search nodes, the Service Account Key JSON should be placed in the same directory as the datanode.conf file
• In Datanode.conf, for all search nodes, specify the required keys in the datanode.conf files
  For s3: s3_client_default_access_key , s3_client_default_secret_key , s3_client_default_endpoint
  For Google Cloud Storage: gcs_credentials_file
  For Network Storage: path_repo
• Restart all search nodes for changes to take effect.
• In Graylog, return to this modal; select the appropriate location from the drop-down.

[tellistone]

As discussed, we'll add this copy to this page in place of an existing banner warning warm tier is not set up.

The option to Create a Warm Tier Repository will not be shown until a Repository Location is configured, which is performed outside of the Graylog UI.

The process for setting up a Warm Tier Repository is documented fully here.

For Opensearch, show the steps as follows:

In summary:

• If planning to use AWS S3 or Google Cloud Storage, first install the corresponding Opensearch Plugin, on all search nodes.
• For a AWS S3 data tiering backend, add your AWS access and secret keys to the OpenSearch keystore on all search nodes - see Opensearch docs on this step here.
• For a Google Cloud Storage Data Tiering backend, add the Service Account Key JSON to the OpenSearch keystore on all search nodes - see Opensearch docs on this step here.
• Make documented changes to the Opensearch configuration file on all search nodes.
• Restart all search nodes.
• In the Graylog UI, return to this page. You should now find that a "Create new warm storage repository" button is shown in place of this warning,

For Data Node, show the steps as follows:

The option to Create a Warm Tier Repository will not be shown until a Repository Location is configured, which is performed outside of the Graylog UI.

The process for setting up a Warm Tier Repository is documented fully here.

In summary:

• If using Google Cloud Storage, on all search nodes, the Service Account Key JSON should be placed in the same directory as the datanode.conf file
• In Datanode.conf, for all search nodes, specify the required keys in the datanode.conf files
  For s3: s3_client_default_access_key , s3_client_default_secret_key , s3_client_default_endpoint
  For Google Cloud Storage: gcs_credentials_file
  For Network Storage: path_repo
• Restart all search nodes for changes to take effect.
• In the Graylog UI, return to this page. You should now find that a "Create new warm storage repository" button is shown in place of this warning,