Does AzureOpenAI provide Batch API by OpenAI

API
1

If I’m correct , the following code will give me error as there is no function as batches .create in AzureOpenAI class yet :
client = AzureOpenAI(
api_key=os.getenv(“AZURE_OPENAI_API_KEY”),
api_version=“2023-12-01-preview”,
azure_endpoint=os.getenv(“AZURE_OPENAI_ENDPOINT”)
)

client.batches.create(
input_file_id=batch_input_file_id,
endpoint=“/v1/chat/completions”,
completion_window=“24h”,
metadata={
“description”: “nightly eval job”
}
)

1 Like
2

Currently, it doesn’t provide the batch API.

3

Could you please help me how to proceed .
I know that there are endpoints one creates through with requests are sent to AzureOpenAI .
Now I will be using OpenAI client only , so do I need to create any such azure endpoint ?

4

Apologies, I am not sure I understand correctly.

5

As mentioned here to use AzureOpenAI , one needs to create key and endpoint first .

But as I cannot use AzureOpenAI for batch API now , so do I need to do all of this .
I will have to use client=OpenAI() , so what are the prerequisite setups I need to do for that .

6

If you want to use the batch API, you need to do that via the OpenAI batch API endpoint using an OpenAI API key from your OpenAI developer account. So in short, there is no point in creating an OpenAI model deployment in Azure if you are looking to use the batch API.

Here are two useful links to get you started:

  1. OpenAI Quickstart Guide with guidance on account and API key set-up (in case you do not yet have a developer account with OpenAI)

  2. Batch guide (also includes a link to the respective API specs)

3 Likes
7

I have to use Open AI APIM like the code given below

 # This code is an example of how to use the OpenAI API with Azure API Management (APIM) in a Jupyter Notebook.
import requests
import json

# Set the parameters
apim_url = "apim_url"
deployment_name = "deployment_name"
api_version = "2024-02-15-preview"
subscription_key = "subscription_key"

# Construct the URL and headers
url = f"{apim_url}/deployments/{deployment_name}/chat/completions?api-version={api_version}"
headers = {
    "Content-Type": "application/json",
    "Ocp-Apim-Subscription-Key": subscription_key
}

# Define the JSON payload
json_payload = {
    "messages": [
        {
            "role": "system",
            "content": "You are an AI assistant that helps people find information."
        },
        {
            "role": "user",
            "content": "What are the differences between Azure Machine Learning and Azure AI services?"
        }
    ],
    "temperature": 0.7,
    "top_p": 0.95,
    "max_tokens": 800
}

# Make the POST request
response = requests.post(url, headers=headers, json=json_payload)

# Print the response text (or you can process it further as needed)
print(response.text)

What changes do I need to make for using the openai Batch API ?

8

There is now a preview version of the batch API on Azure OpenAI. Pick the Python or REST usage method at the top of the page.

2 Likes