Strange Assistants Pricing

API
,
1

Hi everyone,

I’m encountering an issue with unexpected costs while using the assistant APIs.

I’ve created an assistant, and uploaded a PDF file that is 1MB in size. I don’t have any other files or assistants loaded.

Next, I run the following code:

client = openai.OpenAI(api_key="API_KEY")
thread = client.beta.threads.create(
    messages=[
        {
            "role": "user",
            "content": instruction
        }
    ]
)

run = client.beta.threads.runs.create_and_poll(
    thread_id=thread.id,
    assistant_id=assistant_id,
    tool_choice={"type": "file_search"},
    max_completion_tokens=2000
)
messages = list(client.beta.threads.messages.list(thread_id=thread.id, run_id=run.id))

message_content = messages[0].content[0].text

# Output the index
output = {"index": message_content.value}

The goal is to make a single call that generates information about the file and forces the assistant to use file_search.

However, this single call costs me around $0.23, which seems unusually high. Every time I call it, I get charged this amount, no matter what.

I don’t know why is costing me this much.

2

You pay for input to AI models.

gpt-4-turbo AI models = $0.01 per 1000 input tokens.

file search knowlege chunks must be added to API model call input the same as other information you want to ask the AI about, but here, automatically.

file_search places maximum of 20 chunks of 800 tokens + 50% overlap = 16000 + 8000 = 24000 input tokens, the second time the AI is called.

There is no apparent threshold of relevance to limiting top results, a single parameter that could have been employed. The only control is to tell the AI exactly what it will find and not find by invoking the search, so it doesn’t obtain thousands of tokens of tech support knowledge when you ask about penguins.

1 Like