diff --git a/en/docs/design/api-policies/regular-gateway-policies/ai-policies.md b/en/docs/design/api-policies/regular-gateway-policies/ai-policies.md new file mode 100644 index 0000000000..ba23c51337 --- /dev/null +++ b/en/docs/design/api-policies/regular-gateway-policies/ai-policies.md @@ -0,0 +1,38 @@ +# AI Policies + +The following AI Policies can be utilized to cater to your intra-vendor model routing requirements: + +- Model Round Robin Policy +- Model Weighted Robin Robin Policy +- Model Failover Policy + +!!! Note + You can only attach a single AI policy to a given AI API. + +!!! Tip + If you require multiple endpoints to cater to your routing requirements, make sure you have added them under the Endpoints page. + + +## Model Round Robin Policy + +Model Round Robin Policy can be used to route traffic to different models within the same AI/LLM vendor. If the intra-vendor model routing strategy you wish to utilize is round-robin, follow the instructions mentioned below. + +1. Drag and drop the **Model Round Robin** policy which is available under the Common Policies section within the Policy List. + +2. Configure the relevant details under the configuration drawer that appears. Add the production model list, sandbox model list, and specify the suspend duration. + + To add a model, click on **Add Model** button. Then, select a model from the dropdown (if no models are listed, make sure to add the supported model list under the AI/LLM vendor under the Admin Portal). Following the model selection, select the endpoint from the dropdown (if no endpoints are listed, make sure to add the required endpoints under the Endpoints page of Publisher Portal). + +3. Then, click **Save**. + +## Model Weighted Round Robin Policy + +Model Weighted Round Robin Policy can be used to route traffic to different models within the same AI/LLM vendor based on the configured weights. If the intra-vendor model routing stratergy you wish to utilize is weighted round-robin, follow the instructions mentioned below. + +1. Drag and drop the **Model Weighted Round Robin** policy which is available under the Common Policies section within the Policy List. + +2. Configure the relevant details under the configuration drawer that appears. Add the production model list, sandbox model list, and specify the suspend duration. + + To add a model, click on **Add Model** button. Then, select a model from the dropdown (if no models are listed, make sure to add the supported model list under the AI/LLM vendor under the Admin Portal). Following the model selection, select the endpoint from the dropdown (if no endpoints are listed, make sure to add the required endpoints under the Endpoints page of Publisher Portal). Finally, set the desired weight. + +3. Then, click **Save**. diff --git a/en/docs/design/api-policies/revoke-one-time-tokens-policy.md b/en/docs/design/api-policies/regular-gateway-policies/revoke-one-time-tokens-policy.md similarity index 100% rename from en/docs/design/api-policies/revoke-one-time-tokens-policy.md rename to en/docs/design/api-policies/regular-gateway-policies/revoke-one-time-tokens-policy.md diff --git a/en/docs/design/create-api/create-ai-api/create-an-ai-api.md b/en/docs/design/create-api/create-ai-api/create-an-ai-api.md index c4cd1b3c4b..5bca908769 100644 --- a/en/docs/design/create-api/create-ai-api/create-an-ai-api.md +++ b/en/docs/design/create-api/create-ai-api/create-an-ai-api.md @@ -67,7 +67,7 @@ The overview page of the newly created API appears. [![API Key Configurations]({{base_path}}/assets/img/learn/ai-api-auth.png){: style="width:90%"}]({{base_path}}/assets/img/learn/ai-api-auth.png) -3. Click **Save**. +3. Fill out the primary endpoints section by selecting a primary production endpoint and/or a primary sandbox endpoint. Then, click **Save**. Now, you have successfully created an AI API. Next, [deploy the API]({{base_path}}/deploy-and-publish/deploy-on-gateway/deploy-api/deploy-an-api/), [test the API]({{base_path}}/design/create-api/create-rest-api/test-a-rest-api/), and finally [publish the API]({{base_path}}/deploy-and-publish/publish-on-dev-portal/publish-an-api). @@ -75,11 +75,10 @@ Now, you have successfully created an AI API. Next, [deploy the API]({{base_path Learn more on the concepts that you need to know when creating a REST API: - - [API Security]({{base_path}}/design/api-security/api-authentication/secure-apis-using-oauth2-tokens/) - [Rate Limiting for AI APIs]({{base_path}}/design/rate-limiting/rate-limiting-for-ai-apis/) - [Life Cycle Management]({{base_path}}/design/lifecycle-management/api-lifecycle/) - - [API Visibility]({{base_path}}/design/advanced-topics/control-api-visibility-and-subscription-availability-in-developer-portal/) +- [AI Policies]({{base_path}}/design/api-policies/regular-gateway-policies/ai-policies) - [API Documentation]({{base_path}}/design/api-documentation/add-api-documentation/) - [Custom Properties]({{base_path}}/design/create-api/adding-custom-properties-to-apis/) diff --git a/en/mkdocs.yml b/en/mkdocs.yml index abcaf8988a..9565efd759 100644 --- a/en/mkdocs.yml +++ b/en/mkdocs.yml @@ -149,7 +149,8 @@ nav: - Add a Class Mediator: design/api-policies/regular-gateway-policies/adding-a-class-mediator.md - Configure Message Builders and Formatters: design/api-policies/regular-gateway-policies/configuring-message-builders-formatters.md - JWT claim based access validator: design/api-policies/regular-gateway-policies/jwt-claim-based-access-validator.md - - Revoke One Time Tokens Policy: design/api-policies/revoke-one-time-tokens-policy.md + - Revoke One Time Tokens Policy: design/api-policies/regular-gateway-policies/revoke-one-time-tokens-policy.md + - AI Policies: design/api-policies/regular-gateway-policies/ai-policies.md - API Security: - Authentication: - Overview: design/api-security/api-authentication/api-authentication-overview.md