Intermittent response from Code Interpreter

API Feedback
, ,
1

Dear OpenAI API experts,

In my python code, I’m using gpt-4-1106-preview with assistent and code-interpreter as a tool to see if OpenAI API can help to answer questions on structured data with more intelligence than what SQL can do.

data_file_name = "input_data.csv"
assistant_name = "Data Analyst"
assistant_instruction = "You are subject-matter expert in data analysis. When asked to analyse the data, write and run code to answer the question."

…

 #    Upload a file with an "assistants" purpose
 file = client.files.create(

        file=open(data_file_name, "rb"),
         purpose='assistants'
         )

        assistant = client.beta.assistants.create(
        name=assistant_name,
        instructions=assistant_instruction,
        tools=[{"type": "retrieval"},{"type": "code_interpreter"}],
       file_ids=[file.id],
        model="gpt-4-1106-preview"
 )

I then do the prompting in the same thread as assistent

                thread = client.beta.threads.create()
                messages= client.beta.threads.messages.create(
                     thread_id=thread.id,
                     role="user",
                     content=prompt_user)

 

                run = client.beta.threads.runs.create(
                thread_id=thread.id,
                assistant_id=assistentId)

                  polling_run_routine(client, thread, run)

polling_run_routine is just a function that does the retrieval with pooling, attempting every second with timeout after 30 attempts.

 run = client.beta.threads.runs.retrieve(
            thread_id=thread.id,
            run_id=run.id 
          )

Questions/concerns

  1. Intermittent response concerns
    Majority of the calls with prompt as a question on the data results in timeouts and it is very rare to get a response within 1 minute. When it does get back, the answer is pretty accurate though.
    Does any one have the same experience with code-interpreter?
    Am I misusing it to certain extend when trying to make it working with structured data and asking questions that are difficult to translate to a code?
  2. API price
    The volume that I currently pass is very low - around 1K and on around 30 prompt requests with questions such as ‘how many records do you see’, ‘give me duplicate records’ etc, I have already charged around $20 .
    Any ideas of how should I measure the cost ? If I increase the data input in assistent to its acceptable limit - 512MB, will the charge be increased and by how much?

At this point, answer to the first question is more urgent and once I achieve some level of stability in the response, i hopefully can measure the 2nd.

Many thanks in advance,
Lilit