From e1bf3ae81c4d8a6e2e17af24a2163db91c1936a1 Mon Sep 17 00:00:00 2001 From: pablodanswer Date: Tue, 10 Dec 2024 14:12:38 -0800 Subject: [PATCH] update configuration --- resourcing.mdx | 33 ++++++++++++++++++++++----------- 1 file changed, 22 insertions(+), 11 deletions(-) diff --git a/resourcing.mdx b/resourcing.mdx index 19190a5..dcde166 100644 --- a/resourcing.mdx +++ b/resourcing.mdx @@ -6,14 +6,27 @@ icon: "desktop" ## Running Locally -When running locally through Docker, we recommend making at least 4vCPU cores and 10GB of RAM available to Docker -(16GB is preferred). This can be controlled in the **Resources** section of the Docker Desktop settings menu. +When running locally through Docker, we recommend allocating at least 4 vCPU cores and 10GB of RAM to Docker (16GB is preferred). You can adjust these settings in the **Resources** section of the Docker Desktop settings menu. + +To prevent Danswer from consuming excessive resources and potentially impacting your system's performance, you can set resource limits on the Docker containers. This is done by applying the `docker-compose.resources.yml` file when starting Danswer. This file allows you to control and limit the resources used by each Danswer component. + +To apply resource limits: + +1. Use the `docker-compose.resources.yml` file provided in the `deployment/docker_compose` directory. This file contains default resource limits and reservations for each service. + +2. When starting Danswer, include the resource file in your Docker Compose command: + + ```bash + docker-compose -f docker-compose.dev.yml -f docker-compose.resources.yml up + ``` + +3. You can adjust the resource limits by setting environment variables. For more details on available environment variables and default values, refer to the [`docker-compose.resources.yml`](https://github.com/onyx-dot-app/onyx/blob/main/deployment/docker_compose/docker-compose.resources.yml) file. ## Single Cloud Instance We generally recommend setting everything up on a single instance (e.g. an AWS EC2 instance, a Google Compute Engine instance, an Azure VM, etc.) via Docker Compose as it's -the simplest way to get started. For a step-by-step guide on how to do this, checkout our +the simplest way to get started. For a step-by-step guide on how to do this, checkout our [EC2 deployment guide](https://docs.danswer.dev/production/aws/ec2). For most use cases a single reasonably sized instance should be more than enough to guarantee excellent @@ -24,14 +37,14 @@ If you go with this approach, we recommend: - **CPU:** >= 4 vCPU cores (we recommend 8 vCPU cores if possible) - **Memory:** >= 10 GB of RAM (for best performance, 16 is recommended) - - Note: If you are switching embedding models, you will need >= 11 GB RAM as both sets of models need + - Note: If you are switching embedding models, you will need >= 11 GB RAM as both sets of models need to be loaded in simultaneously - **Disk:** >= 50 GB + ~2.5x the size of the indexed documents. Disk is generally very cheap, so we would recommend getting extra disk beyond this recommendation to be safe. - Note: Vespa does not allow writes when disk usage is >75%, so make sure to always have some headroom here - Note: old, unused docker images often take up a bunch of space when performing frequent upgrades of Danswer. - Note: Vespa, used by Danswer for document indexing, requires Haswell (2013) or later CPUs. For older CPUs, use the `vespaengine/vespa-generic-intel-x86_64` image in your `docker-compose.yml`. This generic image is slower but ensures compatibility. For details, see [Vespa CPU Support](https://docs.vespa.ai/en/cpu-support.html). - To clean up these unused images, run: `docker system prune --all`. + To clean up these unused images, run: `docker system prune --all`. For reference, we have chosen to give each Danswer Cloud customer a `m7g.xlarge` instance by default, which has 4vCPU cores + 16GB of RAM + 200GB of disk space. We're comfortable using 4vCPU cores in a @@ -41,7 +54,7 @@ production deployment. ## Kubernetes / AWS ECS -If you prefer to give each component it's own dedicated resources for more efficient +If you prefer to give each component it's own dedicated resources for more efficient scaling, we recommend giving each container access to at least the following resources: `api_server` - 1 CPU, 2Gi Memory @@ -52,11 +65,9 @@ scaling, we recommend giving each container access to at least the following res `postgres` - 500m CPU, 2Gi Memory -`vespa` - >=4 CPU, >= 4Gi Memory. This is the bare minimum, and we would generally recommend -higher than this. The resources required here also scales linearly with the number of documents indexed. -For reference, with 50GB of documents, we would generally recommend at least 10 CPU, 20Gi Memory + +`vespa` - >=4 CPU, >= 4Gi Memory. This is the bare minimum, and we would generally recommend +higher than this. The resources required here also scales linearly with the number of documents indexed. +For reference, with 50GB of documents, we would generally recommend at least 10 CPU, 20Gi Memory + tuning the `VESPA_SEARCHER_THREADS` environment variable. `nginx` - 250m CPU, 128Mi Memory - -