Merge branch 'main' into generate

Author: pamelafox

.devcontainer/devcontainer.json

@@ -16,9 +16,9 @@
             "version": "18",
             "nodeGypDependencies": false
         },
-        "ghcr.io/azure/azure-dev/azd:latest": {},
-        // Required for azd to package the app to ACA
-        "ghcr.io/devcontainers/features/docker-in-docker:2": {},
+        "ghcr.io/azure/azure-dev/azd:latest": {
+			"version": "1.10.1"
+		},
 		"ghcr.io/prulloac/devcontainer-features/ollama:1": {}
     },
 	// Configure tool-specific properties.

.github/workflows/app-tests.yaml

@@ -3,8 +3,18 @@ name: App Tests
 on:
   push:
     branches: [ main ]
+    paths-ignore:
+      - "**.md"
+      - ".azdo/**"
+      - ".devcontainer/**"
+      - ".github/**"
   pull_request:
     branches: [ main ]
+    paths-ignore:
+      - "**.md"
+      - ".azdo/**"
+      - ".devcontainer/**"
+      - ".github/**"
   workflow_dispatch:
 
 permissions:

README.md

@@ -3,15 +3,15 @@
 [![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/Azure-Samples/rag-postgres-openai-python)
 [![Open in Dev Containers](https://img.shields.io/static/v1?style=for-the-badge&label=Dev%20Containers&message=Open&color=blue&logo=visualstudiocode)](https://vscode.dev/redirect?url=vscode://ms-vscode-remote.remote-containers/cloneInVolume?url=https://github.com/azure-samples/rag-postgres-openai-python)
 
-This project creates a web-based chat application with an API backend that can use OpenAI chat models to answer questions about the items in a PostgreSQL database table. The frontend is built with React and FluentUI, while the backend is written with Python and FastAPI.
+This project creates a web-based chat application with an API backend that can use OpenAI chat models to answer questions about the rows in a PostgreSQL database table. The frontend is built with React and FluentUI, while the backend is written with Python and FastAPI.
 
 This project is designed for deployment to Azure using [the Azure Developer CLI](https://learn.microsoft.com/azure/developer/azure-developer-cli/), hosting the app on Azure Container Apps, the database in Azure PostgreSQL Flexible Server, and the models in Azure OpenAI.
 
 * [Features](#features)
 * [Getting started](#getting-started)
-    * [GitHub Codespaces](#github-codespaces)
-    * [VS Code Dev Containers](#vs-code-dev-containers)
-    * [Local environment](#local-environment)
+  * [GitHub Codespaces](#github-codespaces)
+  * [VS Code Dev Containers](#vs-code-dev-containers)
+  * [Local environment](#local-environment)
 * [Deployment](#deployment)
 * [Local development](#local-development)
 * [Costs](#costs)
@@ -27,7 +27,13 @@ This project provides the following features:
 * OpenAI function calling to optionally convert user queries into query filter conditions, such as turning "Climbing gear cheaper than $30?" into "WHERE price < 30".
 * Conversion of user queries into vectors using the OpenAI embedding API.
 
-![Screenshot of chat app with question about climbing gear](docs/screenshot_chat.png)
+![Screenshot of chat app with question about climbing gear](docs/images/screenshot_chat.png)
+
+## Architecture diagram
+
+The deployed app uses a user-assigned managed identity to authenticate to Azure services, and stores logs in Log Analytics.
+
+![Architecture diagram: Azure Container Apps, Azure Container Registry, Managed Identity, Azure OpenAI, Azure Database for PostgreSQL](docs/images/azure_architecture.png)
 
 ## Getting started
 
@@ -95,7 +101,11 @@ Once you've opened the project in [Codespaces](#github-codespaces), [Dev Contain
     azd auth login
     ```
 
-    If you have any issues with that command, you may also want to try `azd auth login --use-device-code`.
+    For GitHub Codespaces users, if the previous command fails, try:
+
+   ```shell
+    azd auth login --use-device-code
+    ```
 
 2. Create a new azd environment:
 
@@ -107,7 +117,7 @@ Once you've opened the project in [Codespaces](#github-codespaces), [Dev Contain
 
 3. (Optional) If you would like to customize the deployment to [use existing Azure resources](docs/deploy_existing.md), you can set the values now.
 
-3. Provision the resources and deploy the code:
+4. Provision the resources and deploy the code:
 
     ```shell
     azd up
@@ -136,7 +146,7 @@ Since the local app uses OpenAI models, you should first deploy it for the optim
 1. Run these commands to install the web app as a local package (named `fastapi_app`), set up the local database, and seed it with test data:
 
     ```bash
-    python3 -m pip install -e src/backend
+    python -m pip install -e src/backend
     python ./src/backend/fastapi_app/setup_postgres_database.py
     python ./src/backend/fastapi_app/setup_postgres_seeddata.py
     ```
@@ -155,7 +165,7 @@ Since the local app uses OpenAI models, you should first deploy it for the optim
 3. Run the FastAPI backend (with hot reloading). This should be run from the root of the project:
 
     ```shell
-    python3 -m uvicorn fastapi_app:create_app --factory --reload
+    python -m uvicorn fastapi_app:create_app --factory --reload
     ```
 
     Or you can run "Backend" in the VS Code Run & Debug menu.
@@ -191,7 +201,10 @@ Additionally, we have added a [GitHub Action](https://github.com/microsoft/secur
 
 Further documentation is available in the `docs/` folder:
 
+* [Understanding the RAG flow](docs/rag_flow.md)
+* [Customizing the data](docs/customize_data.md)
 * [Deploying with existing resources](docs/deploy_existing.md)
+* [Using Entra auth with PostgreSQL tools](docs/using_entra_auth.md)
 * [Monitoring with Azure Monitor](docs/monitoring.md)
 * [Load testing](docs/loadtesting.md)
 * [Evaluation](docs/evaluation.md)
@@ -200,5 +213,6 @@ Please post in the issue tracker with any questions or issues.
 
 ## Resources
 
+* [RAGHack livestream: Building RAG with PostgreSQL](https://www.youtube.com/watch?v=Dk65oQjYAfo)
 * [RAG chat with Azure AI Search + Python](https://github.com/Azure-Samples/azure-search-openai-demo/)
 * [Develop Python apps that use Azure AI services](https://learn.microsoft.com/azure/developer/python/azure-ai-for-python-developers)

azure.yaml

@@ -3,13 +3,15 @@ name: rag-postgres-openai-python
 metadata:
     template: rag-postgres-openai-python@0.0.1
 requiredVersions:
-    azd: ">= 1.9.5"
+    azd: ">= 1.10.0"
 services:
     web:
         project: ./src/backend
         language: py
         module: web
         host: containerapp
+        docker:
+            remoteBuild: true
         hooks:
             prebuild:
                 windows:

docs/README.md

@@ -16,7 +16,7 @@ page_type: sample
 urlFragment: rag-postgres-openai-python
 ---
 
-This project creates a web-based chat application with an API backend that can use OpenAI chat models to answer questions about the items in a PostgreSQL database table. The frontend is built with React and FluentUI, while the backend is written with Python and FastAPI.
+This project creates a web-based chat application with an API backend that can use OpenAI chat models to answer questions about the rows in a PostgreSQL database table. The frontend is built with React and FluentUI, while the backend is written with Python and FastAPI.
 
 This project is designed for deployment to Azure using [the Azure Developer CLI](https://learn.microsoft.com/azure/developer/azure-developer-cli/), hosting the app on Azure Container Apps, the database in Azure PostgreSQL Flexible Server, and the models in Azure OpenAI.
 

docs/customize_data.md

@@ -0,0 +1,57 @@
+# RAG on PostgreSQL: Customizing the data
+
+This guide shows you how to bring in a table with a different schema than the sample table.
+For a full example of code changes needed, check out [this branch](https://github.com/Azure-Samples/rag-postgres-openai-python/compare/main...otherdata).
+
+## Define the table schema
+
+1. Update seed_data.json file with the new data
+2. Update the SQLAlchemy models in postgres_models.py to reflect the new schema
+3. Add the new table to the database:
+
+    ```shell
+    python src/backend/fastapi_app/setup_postgres_database.py
+    ```
+
+    These scripts will run on the local database by default. They will run on the production database as part of the `azd up` deployment process.
+
+## Add embeddings to the seed data
+
+If you don't yet have any embeddings in `seed_data.json`:
+
+1. Update the references to models in update_embeddings.py
+2. Generate new embeddings for the seed data:
+
+    ```shell
+    python src/backend/fastapi_app/update_embeddings.py  --in_seed_data
+    ```
+
+    That script will use whatever OpenAI host is defined in the `.env` file.
+    You may want to run it twice for multiple models, once for Azure OpenAI embedding model and another for Ollama embedding model. Change `OPENAI_EMBED_HOST` between runs.
+
+## Add the seed data to the database
+
+Now that you have the new table schema and `seed_data.json` populated with embeddings, you can add the seed data to the database:
+
+    ```shell
+    python src/backend/fastapi_app/setup_postgres_seeddata.py
+    ```
+
+## Update the LLM prompts
+
+3. Update the question answering prompt at `src/backend/fastapi_app/prompts/answer.txt` to reflect the new domain.
+4. Update the function calling definition in `src/backend/fastapi_app/query_rewriter.py` to reflect the new schema and domain. Replace the `brand_filter` and `price_filter` with any filters that are relevant to your new schema.
+5. Update the query rewriting prompt at `src/backend/fastapi_app/prompts/query.txt` to reflect the new domain and filters.
+6. Update the query rewriting examples at `src/backend/fastapi_app/prompts/query_fewshots.json` to match the new domain and filters.
+
+## Update the API
+
+The FastAPI routes use type annotations to define the schema of the data that they accept and return, so you'll need to update the API to reflect the new schema.
+
+1. Modify `ItemPublic` in `src/backend/fastapi_app/api_models.py` to reflect the new schema.
+2. Modify `RAGContext` if your schema uses a string ID instead of integer ID.
+
+## Update the frontend
+
+1. Modify the Answer component in `src/frontend/src/components/Answer/Answer.tsx` to display the desired fields from the new schema.
+2. Modify the examples in `/workspace/src/frontend/src/components/Example/ExampleList.tsx` to examples for the new domain.

docs/deploy_existing.md

@@ -1,9 +1,9 @@
-# Deploying with existing Azure resources
+# RAG on PostgreSQL: Deploying with existing Azure resources
 
 If you already have existing Azure resources, or if you want to specify the exact name of new Azure Resource, you can do so by setting `azd` environment values.
 You should set these values before running `azd up`. Once you've set them, return to the [deployment steps](../README.md#deployment).
 
-### Openai.com OpenAI account
+## Openai.com OpenAI account
 
 1. Run `azd env set DEPLOY_AZURE_OPENAI false`
 1. Run `azd env set OPENAI_CHAT_HOST openaicom`

docs/images/advanced_hybrid_flow.png

@@ -1,4 +1,4 @@
-## Load testing
+# RAG on PostgreSQL: Load testing
 
 We recommend running a loadtest for your expected number of users.
 You can use the [locust tool](https://docs.locust.io/) with the `locustfile.py` in this sample

docs/images/azure_architecture.png

@@ -1,5 +1,4 @@
-
-# Monitoring with Azure Monitor
+# RAG on PostgreSQL: Monitoring with Azure Monitor
 
 By default, deployed apps use Application Insights for the tracing of each request, along with the logging of errors.
 

docs/images/function_calling_diagram.png

@@ -0,0 +1,61 @@
+# RAG on PostgreSQL: Understanding the RAG flow
+
+This RAG application uses an LLM to answer questions based off matching rows in a PostgreSQL database. It supports two different RAG flows:
+
+* [Simple RAG flow](#simple-rag-flow): First searches the database for a matching row, then uses the LLM to generate an answer.
+* [Advanced RAG flow](#advanced-rag-flow): Adds a query rewriting step before searching the database.
+
+## Simple RAG flow
+
+The simple flow is only used if you de-select the "Advanced flow" option in the Developer Settings dialog. We do not recommend this flow for production use, as it is less sophisticated, but it may be helpful for development and comparison.
+
+This diagram shows the simple RAG flow:
+
+![Simple RAG flow diagram](images/simple_hybrid_flow.png)
+
+1. The user asks a question.
+2. The app uses an embedding model to generate a vector embedding for the question.
+3. The app searches the target table in the PostgreSQL database using a hybrid search (full-text search plus vector search) to find the most relevant rows.
+4. The app sends the original user question and the retrieved rows to the LLM to generate an answer.
+
+When using the app, click the lightbulb on an answer to see the steps:
+
+![Simple RAG flow example](images/simple_rag_thoughts.png)
+
+For the full code, see [/src/backend/fastapi_app/rag_simple.py](/src/backend/fastapi_app/rag_simple.py).
+
+## Advanced RAG flow
+
+The advanced flow is used if you select the "Advanced flow" option in the Developer Settings dialog, and is the default flow for the RAG application.
+
+This diagram shows the advanced RAG flow:
+
+![Advanced RAG flow diagram](images/advanced_hybrid_flow.png)
+
+The difference between the simple and advanced flows is an additional query rewriting step before the database search, which can improve the relevance of the search results. All the steps:
+
+1. The user asks a question.
+2. The app uses an LLM to rewrite the question into a text query that can be used to search the database.
+3. The app uses an embedding model to generate a vector embedding for the rewritten text query.
+4. The app searches the target table in the PostgreSQL database using a hybrid search (full-text search plus vector search) to find the most relevant rows.
+5. The app sends the original user question and the retrieved rows to the LLM to generate an answer.
+
+When using the app, click the lightbulb on an answer to see the query rewriting step call and its results.
+
+![Query rewriting example](images/query_rewriting_thoughts.png)
+
+For the full code, see [/src/backend/fastapi_app/rag_advanced.py](/src/backend/fastapi_app/rag_advanced.py).
+
+### Query rewriting with function calling
+
+Let's dive even deeper into the query rewriting step.
+
+The query rewriting step uses [OpenAI function calling](https://platform.openai.com/docs/guides/function-calling) to generate the rewritten query. That function may also optionally specify column filters, which are then applied to the SQL query for the search step. For the sample data, the filters are for the "brand" and "price" columns, but you could change those to match your own [custom data schema](customize_data.md).
+
+This diagram illustrates the function calling process for a sample question:
+
+![Function calling diagram](images/function_calling_diagram.png)
+
+The `search_database` function definition is in [query_rewriter.py](/src/backend/fastapi_app/query_rewriter.py), and the few shot examples are in [query_fewshots.json](/src/backend/fastapi_app/prompts/query_fewshots.json). The function calling response is parsed in [query_rewriter.py](/src/backend/fastapi_app/query_rewriter.py) to extract the suggested SQL query and column filters, and those are passed to [postgres_searcher.py](/src/backend/fastapi_app/postgres_searcher.py) to search the database.
+
+To be able to use function calling, the app must use a model that has support for it. The OpenAI GPT models do support function calling, but other models may not. If you're developing locally with Ollama, we recommend llama3.1 as it has been tested to work with function calling.