diff --git a/.markdownlint.yaml b/.markdownlint.yaml new file mode 100644 index 0000000..8589cec --- /dev/null +++ b/.markdownlint.yaml @@ -0,0 +1,3 @@ +--- +MD013: + code_blocks: false diff --git a/service-catalog/README.md b/service-catalog/README.md new file mode 100644 index 0000000..910441a --- /dev/null +++ b/service-catalog/README.md @@ -0,0 +1,15 @@ +# Service Catalog + +The service catalog lists the infrastructure, services, and apps that the +maintains. It is a work-in-progress and not exhaustive. It is our intention to +improve it as we go. + +The documentation in the service catalog is inspired by the [Diataxis] +framework, which proposes a systematic approach to categorize into tutorials, +how-to guides, explanations, and reference documentation. + +## External Services + +- [Fastly](./fastly/README.md) + +[diataxis]: https://diataxis.fr/ diff --git a/service-catalog/fastly/README.md b/service-catalog/fastly/README.md new file mode 100644 index 0000000..f1760b6 --- /dev/null +++ b/service-catalog/fastly/README.md @@ -0,0 +1,11 @@ +# Fastly + +[Fastly] is a [Content Delivery Network] (CDN) that caches content and makes it +available closer to the user. + +## How-to Guides + +- [How to test cache invalidations](./how-to-test-cache-invalidations.md) + +[content delivery network]: https://en.wikipedia.org/wiki/Content_delivery_network +[fastly]: https://www.fastly.com/ diff --git a/service-catalog/fastly/how-to-test-cache-invalidations.md b/service-catalog/fastly/how-to-test-cache-invalidations.md new file mode 100644 index 0000000..e54c858 --- /dev/null +++ b/service-catalog/fastly/how-to-test-cache-invalidations.md @@ -0,0 +1,137 @@ +# How to Test Cache Invalidations + +We use [Fastly] to cache Rust releases and crates. Both are cached with a long +time-to-live (TTL) to improve the cache hit ratio and reduce our costs for +outbound traffic from S3. However, we need to invalidate the cache when new +versions of Rust or crates are released. This is done through Fastly's API as +part of the release and publish processes. + +Sometimes, it is necessary to test whether the cache invalidation works as +expected. This document outlines the steps that need to be taken. + +## Prerequisites + +## Find an Artifact + +The first step is finding a suitable artifact for the test. + +1. Go to and log into the AWS console. +2. Open the legacy account (`Rust Admin - 8450`) and navigate to the S3 service. +3. Open the `dev-static-rust-lang-org` bucket and go into the `rustup` folder. +4. Find a file that is not frequently updated, e.g. one in the archive. + +For the rest of the guide, we will be working with the checksum file +`/rustup/archive/1.27.0/x86_64-unknown-linux-gnu/rustup-init.sha256`. + +## Download the Artifact + +The second step is downloading the artifact to ensure it is in the cache. This +can be done by fetching the file from the Fastly service: + +```shell +curl -I https://fastly-dev-static.rust-lang.org/rustup/archive/1.27.0/x86_64-unknown-linux-gnu/rustup-init.sha256 +``` + +Make sure that the `x-cache` header is a `HIT`. This indicates that the file is +cached in Fastly's network. Check the `age` header and note down the age of the +file. + +After invalidating the cache, we expct the `x-cache` header to be a `MISS` and +the `age` header to be `0`. + +## Purge the Cache + +The third step is purging the cache. We use two different methods to purge the +cache: [surrogate keys] for Rust releases and [URL purges] for crates. Follow +the respective instructions below. + +### Purge a Surrogate Key + +Purging a surrogate key invalidates all objects that are tagged with the key. +Which objects get tagged with which key is defined in the VCL configuration for +the Fastly service, which can be found in [`terragrunt/modules/release-distribution/fastly-static.tf`] +in [rust-lang/simpleinfra]. + +For this example, we will be purging a rustup artifact using the `rustup` key. + +For convenience, we export the Fastly service ID as an environment variable. +Change this if you are working on a different service or environment. + +```shell +# dev-static.rust-lang.org +export FASTLY_SERVICE_ID=5qaYFyyiorVua6uCZg7It0 +``` + +We also need a Fastly authentication token. Replace `` with +the actual token. + +```shell +export FASTLY_AUTH_TOKEN= +``` + +With a valid authentication token from Fastly, use the following command to send +a [purge request](https://www.fastly.com/documentation/reference/api/purging/#purge-tag) +for the key. + +```shell +curl -i -X POST "https://api.fastly.com/service/${FASTLY_SERVICE_ID}/purge/rustup" \ + -H "Fastly-Key: ${FASTLY_AUTH_TOKEN}" \ + -H "Accept: application/json" +``` + +We expect the response to have a `200 OK` status code and return an ID for the +purge in its body: + +```json +{ "status": "ok", "id": "8230126-1715084554-7370607" } +``` + +### Purge a URL + +The simplest way to purge an object on Fastly is by purging its URL. + +Export the URL that you want to purge as an environment variable: + +```shell +export URL_TO_PURGE=fastly-dev-static.rust-lang.org/rustup/archive/1.27.0/x86_64-unknown-linux-gnu/rustup-init.sha256 +``` + +We also need a Fastly authentication token. Replace `` with +the actual token and export it as an environment variable for convenience. + +```shell +export FASTLY_AUTH_TOKEN= +``` + +With a valid authentication token from Fastly, use the following command to send +a [purge request](https://www.fastly.com/documentation/reference/api/purging/#purge-single-url) +for the URL. + +```shell +curl -i -X POST "https://api.fastly.com/purge/${URL_TO_PURGE}" \ + -H "Fastly-Key: ${FASTLY_AUTH_TOKEN}" \ + -H "Accept: application/json" +``` + +We expect the response to have a `200 OK` status code and return an ID for the +purge in its body: + +```json +{ "status": "ok", "id": "8230126-1715084554-7370607" } +``` + +## Verify Purge + +After purging the cache, we can verify that the cache has been invalidated by +fetching the file again: + +```shell +curl -I https://fastly-dev-static.rust-lang.org/rustup/archive/1.27.0/x86_64-unknown-linux-gnu/rustup-init.sha256 +``` + +The `x-cache` header should now be a `MISS` and the `age` header should be `0`. + +[`terragrunt/modules/release-distribution/fastly-static.tf`]: https://github.com/rust-lang/simpleinfra/blob/master/terragrunt/modules/release-distribution/fastly-static.tf +[rust-lang/simpleinfra]: https://github.com/rust-lang/simpleinfra +[surrogate keys]: https://www.fastly.com/documentation/guides/concepts/edge-state/cache/purging/#surrogate-key-purge +[url purges]: https://www.fastly.com/documentation/guides/concepts/edge-state/cache/purging/#url-purge