Also consider cognitiveservices.azure.com an AzureEndpoint #470
Conversation
| @@ -93,9 +93,11 @@ internal fun Any?.contentToString(): String { | |||
| internal fun isAzureEndpoint(baseUrl: String): Boolean { | |||
| // Azure Endpoint should be in the format of `https://<region>.openai.azure.com`. | |||
| // Or `https://<region>.azure-api.net` for Azure OpenAI Management URL. | |||
| // Or `<user>-random-<region>.cognitiveservices.azure.com`. | |||
There was a problem hiding this comment.
| // Or `<user>-random-<region>.cognitiveservices.azure.com`. | |
| // Or `<user>-random-<region>.cognitiveservices.azure.com` for deployments across regions. |
this is my best guess regarding the reason that I got this URL for this deployment, but I cannot know for sure, if reason is optional I would prefer leaving it empty
There was a problem hiding this comment.
| // Or `<user>-random-<region>.cognitiveservices.azure.com`. | |
| // Or `<user>-random-<region>.cognitiveservices.azure.com` for Azure AI services deployments. |
It might also be that reason, my openai deployment has the openai.azure.com url, but my azure ai service deployment has cognitiveservices.azure.com
|
@jpalvarezl I'm not sure what's the right thing to do here Can you weigh in on this PR and ticket? #469 |
|
Hmmm, wait, should the environment variable being set to that URL result in the same behavior as what this example is doing without an environment variable? |
|
Although, maybe that's infeasible because we don't want to reference Azure dependencies from the @StefanLobbenmeierObjego does doing it the way the example shows solve this for you? |
|
No this example does not help me: import com.azure.identity.AuthenticationUtil;
import com.azure.identity.DefaultAzureCredentialBuilder;
import com.openai.client.OpenAIClient;
import com.openai.credential.BearerTokenCredential;
import com.openai.models.ChatModel;
import com.openai.models.chat.completions.ChatCompletionCreateParams;
import static com.openai.client.okhttp.OpenAIOkHttpClient.*;
class Scratch {
public static void main(String[] args) {
assert System.getenv("AZURE_OPENAI_KEY") != null;
assert System.getenv("OPENAI_BASE_URL") != null;
assert System.getenv("OPENAI_BASE_URL").endsWith("cognitiveservices.azure.com");
OpenAIClient client = builder()
// Gets the API key from the `AZURE_OPENAI_KEY` environment variable
.fromEnv()
// Set the Azure Entra ID
.credential(BearerTokenCredential.create(AuthenticationUtil.getBearerTokenSupplier(
new DefaultAzureCredentialBuilder().build(), "https://cognitiveservices.azure.com/.default")))
.build();
ChatCompletionCreateParams createParams = ChatCompletionCreateParams.builder()
.model(ChatModel.GPT_3_5_TURBO)
.maxCompletionTokens(2048)
.addDeveloperMessage("Make sure you mention Stainless!")
.addUserMessage("Tell me a story about building the best SDK!")
.build();
client.chat().completions().create(createParams).choices().stream()
.flatMap(choice -> choice.message().content().stream())
.forEach(System.out::println);
}
} |
Slightly confused by this comment, fromEnv() does not actually persist anything about if this is an azure endpoint or an openai endpoint: openai-java/openai-java-core/src/main/kotlin/com/openai/core/ClientOptions.kt Lines 237 to 259 in b8fe68a The only way right now to get the "openai" and "deployment" into the path is by having isAzureEndpoint true here: And this is what I am trying to achieve with this PR. A possible alternative could be to introduce "isAzureEndpoint" to the client options, which could be set to true when the URL matches or when "AZURE_OPENAI_KEY" env is set, but thats a bigger change so I would prefer to stick with the url matching kind. |
Fixes #469