Get Started with Chat Using Azure AI Foundry

MENU: PREREQUISITES | DEVELOPMENT | DEPLOYMENT | TRACING AND MONITORING | DEVELOPMENT OPTIONS | SUPPORTING DOCUMENTATION

This solution contains a simple chat application that is deployed to Azure Container Apps.This solution also creates an Azure AI Foundry hub, project and connected resources including Azure AI Services, AI Search and more. The main path walks you through deploying the application using local development environment. After the local development environment instructions, there are also instructions for developing in GitHub Codespaces and VS Code Dev Containers.

Architecture diagram

Prerequisites

Azure account

If you do not have an Azure Subscription, you can sign up for a free Azure account and create an Azure Subscription.

NOTE! Whether you sign up for a new account, or use an existing subscription, check that you have the necessary permissions:

• Your Azure account for the Azure Subscription that you are using must have Microsoft.Authorization/roleAssignments/write permissions, such as Role Based Access Control Administrator, User Access Administrator, or Owner.
• Your Azure account also needs Microsoft.Resources/deployments/write permissions on the subscription level.

You can view the permissions for your account and subscription by going to Azure portal, clicking 'Subscriptions' under 'Navigation' and then choosing your subscription from the list. If you try to search for your subscription and it does not come up, make sure no filters are selected. After selecting your subscription, select 'Access control (IAM)' and you can see the roles that are assigned to your account for this subscription. If you want to see more information about the roles, you can go to the 'Role assignments' tab and search by your account name and then click on the role you want to view more information about.

Required tools

Make sure the following tools are installed:

1. Azure Developer CLI (azd) Install or update to the latest version. Instructions can be found on the linked page.
2. Python 3.9+
3. Git

Development

Code

Download the project code:

git clone https://github.com/Azure-Samples/azureai-basic-python.git

At this point you could make changes to the code if required. However, no changes are needed to deploy and test the app as shown in the next step.

Logging

If you want to enable logging to a file, uncomment the following line in Dockerfile located in the src directory:

ENV APP_LOG_FILE=app.log

By default the file name app.log is used. You can provide your own file name by replacing app.log with the desired log file name.

NOTE! Any changes to the Dockerfile require a re-deployment in order for the changes to take effect.

The provided file logging implementation is intended for development purposes only, specifically for testing with a single client/worker. It should not be used in production environments after the R&D phase.

Tracing to Azure Monitor

To enable tracing to Azure Monitor, modify the value of ENABLE_AZURE_MONITOR_TRACING environment variable to true in Dockerfile found in src directory:

ENV ENABLE_AZURE_MONITOR_TRACING=true

Note that the optional App Insights resource is required for tracing to Azure Monitor (it is created by default).

To enable message contents to be included in the traces, set the following environment variable to true in the same Dockerfile. Note that the messages may contain personally identifiable information.

ENV AZURE_TRACING_GEN_AI_CONTENT_RECORDING_ENABLED=true

Retrieval-Augmented Generation model

Retrieval-Augmented Generation (RAG) technology is based on use of similarity search to find the appropriate context for the question asked by user. This feature is optional, and it is turned on by default. Please set USE_SEARCH_SERVICE environment variable to false to turn if off, in this case the Azure AI Search resource will not be created. If the RAG feature is on, we perform the vector search to find the context, and if it is not found, the naive response from the LLM model is being returned. The source code comes with the data set about hiking products as an example. This data set was split by sentences, which were used to build embeddings using OpenAI text-embedding-3-small model with dimensions parameter equals to 100, resulting in embeddings.csv file, located in folder api/data. To train the model with different data set, or different model, please run the script:

from .api.search_index_manager import SearchIndexManager

search_index_manager = SearchIndexManager (
    endpoint=self.search_endpoint,
    credential=your_credentials,
    index_name=index_name,
    dimensions=100,
    model="text-embedding-3-small",
    embeddings_client=embedding_client,
)
await search_index_manager.build_embeddings_file(
    input_directory='data',
    output_file='data/embeddings.csv'
    sentences_per_embedding=4
)

Please note the sentences_per_embedding parameter, which specifies the number of sentences used to construct the embedding. The larger this number, the broader the context that will be identified during the similarity search. For each question asked from the application, we first search the answer in vector store and if the answer was found, we return the response based on data provided in the data set. To deploy the application with RAG model, please provide the next environment variables:

USE_SEARCH_SERVICE=true
AZURE_AI_SEARCH_INDEX_NAME=index_sample
AZURE_AI_EMBED_DEPLOYMENT_NAME=text-embedding-3-small

If these variables, except USE_SEARCH_SERVICE, which is true by default, were not set, or there is no Azure AI Search connection, RAG search will not be used. In this application we are creating index, called index_sample in Azure Search Service. The index can be created by following instructions, or by calling the next code:

search_index_manager.ensure_index_created(vector_index_dimensions)
search_index_manager.upload_documents(embeddings_path)

In this case vector_index_dimensions needs to be provided only if dimensions were not set during SearchIndexManager object creation. If the index will be present before the aplication deployment, application will skip document upload and will use the existing index.

Deployment

1. Login to Azure:

   azd auth login

2. (Optional) If you would like to customize the deployment to disable resources, customize resource names, or customize the models, you can follow those steps now.

3. Provision and deploy all the resources by running the following in azureai-basic-python directory:

   azd up

   It will prompt you to provide an azd environment name (like "azureaiapp"), select a subscription from your Azure account, and select a location which has quota for all the resources. Then it will provision the resources in your account and deploy the latest code. If you get an error or timeout with deployment, changing the location can help, as there may be availability constraints for the resources.

   NOTE! If you get authorization failed and/or permission related errors during the deployment, please refer to the Azure account requirements in the Prerequisites section.

4. When azd has finished deploying, you'll see an endpoint URI in the command output. Visit that URI, and you should see the app! 🎉 You can view information about your deployment with:

   azd show

5. If you make further modification to the app code, you can deploy the updated version with:

   azd deploy

   You can get more detailed output with the --debug parameter.

   azd deploy --debug

   Check for any errors during the deployment, since updated app code will not get deployed if errors occur.

⚠️ To avoid unnecessary costs, remember to take down your app if it's no longer in use, either by deleting the resource group in the Portal or running azd down.

Tracing and Monitoring

You can view console logs in Azure portal. You can get the link to the resource group with the azd tool:

azd show

Or if you want to navigate from the Azure portal main page, select your resource group from the 'Recent' list, or by clicking the 'Resource groups' and searching your resource group there.

After accessing you resource group in Azure portal, choose your container app from the list of resources. Then open 'Monitoring' and 'Log Stream'.

If you enabled logging to a file, you can view the log file by choosing 'Console' under the 'Monitoring' (same location as above for the console traces), opening the default console and then for example running the following command (replace app.log with the actual name of your log file):

more app.log

You can view the App Insights tracing in Azure AI Foundry. Select your project on the Azure AI Foundry page and then click 'Tracing'.

Development Options

In addition to the development setup documented above, the following development environment options can be used.

Local Development Server

Local Development Server

You can optionally use a local development server to test app changes locally. Make sure you first deployed the app to Azure before running the development server.

1. Create a Python virtual environment and activate it.

   On Windows:

   python -m venv .venv
   .venv\scripts\activate

   On Linux:

   python3 -m venv .venv
   source .venv/bin/activate

2. Navigate to the src directory:

   cd src

3. Install required Python packages:

   python -m pip install -r requirements.txt

4. Run the local server:

   python -m uvicorn "api.main:create_app" --factory --reload

5. Click 'http://127.0.0.1:8000' in the terminal, which should open a new tab in the browser.

6. Enter your message in the box.

GitHub Codespaces

Deploy in GitHub Codespaces

You can run this template virtually by using GitHub Codespaces. The button will open a web-based VS Code instance in your browser:

1. Open the template (this may take several minutes):

   

2. Open a terminal window

3. Continue with the deploying steps

VS Code Dev Containers

VS Code Dev Containers

A related option is VS Code Dev Containers, which will open the project in your local VS Code using the Dev Containers extension:

1. Start Docker Desktop (install it if not already installed Docker Desktop)

2. Open the project:

   

3. In the VS Code window that opens, once the project files show up (this may take several minutes), open a terminal window.

4. Continue with the deploying steps

Supporting Documentation

Costs

Pricing varies per region and usage, so it isn't possible to predict exact costs for your usage. The majority of the Azure resources used in this infrastructure are on usage-based pricing tiers. However, Azure Container Registry has a fixed cost per registry per day.

You can try the Azure pricing calculator for the resources:

• Azure AI Foundry: Free tier. Pricing
• Azure AI Search: Standard tier, S1. Pricing is based on the number of documents and operations. Pricing
• Azure Storage Account: Standard tier, LRS. Pricing is based on storage and operations. Pricing
• Azure Key Vault: Standard tier. Pricing is based on the number of operations. Pricing
• Azure AI Services: S0 tier, defaults to gpt-4o-mini and text-embedding-ada-002 models. Pricing is based on token count. Pricing
• Azure Container App: Consumption tier with 0.5 CPU, 1GiB memory/storage. Pricing is based on resource allocation, and each month allows for a certain amount of free usage. Pricing
• Azure Container Registry: Basic tier. Pricing
• Log analytics: Pay-as-you-go tier. Costs based on data ingested. Pricing

⚠️ To avoid unnecessary costs, remember to take down your app if it's no longer in use, either by deleting the resource group in the Portal or running azd down.

Security guidelines

This template uses Azure AI Foundry connections to communicate between resources, which stores keys in Azure Key Vault. This template also uses Managed Identity for local development and deployment.

To ensure continued best practices in your own repository, we recommend that anyone creating solutions based on our templates ensure that the Github secret scanning setting is enabled.

You may want to consider additional security measures, such as:

• Enabling Microsoft Defender for Cloud to secure your Azure resources.
• Protecting the Azure Container Apps instance with a firewall and/or Virtual Network.

Resources

This template creates everything you need to get started with Azure AI Foundry:

• AI Hub Resource
• AI Project
• Azure AI Service: Default models deployed are gpt-4o-mini and text-embedding-ada-002, but any Azure AI models can be specified per the documentation.
• AI Search Service (Optional, disabled by default)

The template also includes dependent resources required by all AI Hub resources:

• Storage Account
• Key Vault
• Application Insights (Optional, enabled by default)
• Container Registry (Optional, enabled by default)