Unusually high API bill from Assistants API

API
1

So I’m trying to implement a data analyst assistant for my project. The issue is that just running this one time cost me $0.66. The raw json in this case checks out at about 5561 tokens according to this online tool here: GPT-4 Token Calculator - Calculate Your Total GPT-4 Tokens and GPT-4 Token Price - Works for GPT-4, GPT-4 Turbo, GPT-4V, and GPT-4 Vision with text and images.

According to my OpenAI account on API usage:

context tokens used: 62000
generated tokens: 1300

What gives? Is there something wrong with my api route handler in next js? Before switching to this preview model, I was running my thread on “gpt4” model (which may not support assistants?) and that cost me a few dollars somehow, I had to delete the assistant to stop it rising any further.

Code in snippet below:

export default async function handler(req: NextApiRequest, res: NextApiResponse) {
  try {
    const dataAnalyst = await findOrCreateDataAnalyst()
    const run = await openai.beta.threads.createAndRun({
      assistant_id: dataAnalyst.id,
      thread: {
        messages: [
          {
            role: 'user',
            content: `Generate an in depth data analysis of the following data:\n\n${JSON.stringify(req.body.data, null, 2)}`,
          },
        ],
      },
    })
    let runStatus = await openai.beta.threads.runs.retrieve(run.thread_id, run.id)

    while (runStatus.status !== 'completed') {
      if (runStatus.status === 'failed' || runStatus.status === 'cancelled') {
        throw new Error(`The run has ended in a non-successful status: ${runStatus.status}`)
      }

      await new Promise((resolve) => setTimeout(resolve, 2000))
      runStatus = await openai.beta.threads.runs.retrieve(run.thread_id, run.id)
    }

    const messages = await openai.beta.threads.messages.list(run.thread_id)
    res.status(200).json({ messages })
  } catch (error) {
    console.error('Error in API handler:', error)
    res.status(500).json({ error: 'An unexpected error occurred while processing your request.' })
  }
}

Is there something wrong with my code or is the assistants API just really this expensive?

3

the code looks about right to me. What does your creation code look like. Do you have file retrievals?

I do agree assistant api with gpt4 is way too expensive tho. I had 17 page pdf and 1 query cost me 0.15