Data File Analysis via API

API
,
1

Objective: I aim to utilize AI for data analysis from our company system.
Scenario: The user has access to AI directly within our company’s system environment. Communication with OpenAI is exclusively through the API. The user should be able to ask, “What tasks is Paul working on?” and receive an accurate response from the AI.

Approach:

  • I initiate a call to an AI assistant - The assistant has a defined function for data retrieval, which it calls.
  • Retrieve data from my database - Based on this function, I export the underlying data (such as tasks from the system) and subsequently upload it to the assistant (I’ve tested the json and xlsx formats). The source files have about 300 rows and 15 columns in my testing.
  • The assistant’s response is then relayed back to the user.
  • Alongside the file, a description of the table columns is provided to the assistant for better navigation.

Used AI models: ChatGPT 4 and ChatGPT4 Turbo models in the assistant, which have shown similar outcomes.

Problems Encountered:

  1. Long Processing Time: Minimum of 60 seconds on the Assistant’s side, sometimes extending to 5 minutes (exclusive of data export time).
  2. High Token Consumption: Ranges between 4,000 to 30,000 tokens per question.
  3. If I ask the same question 5 times, the AI responds correctly 3 times and 2 times it responds that it cannot retrieve data from my system at all.

Request for Assistance: I am seeking guidance from anyone who has tackled a similar scenario on how to efficiently and effectively manage this use-case.

Thank you very much for the answers

2 Likes
2

I am facing the same issue

Earlier it used to give response within a minute but now it is pauses in the middle of the api call and I receive the respoonse after 10 mins.

Its been one week and still there is no improvement.

1 Like
3

Unfortunately, your first two problems are persistent issues with the Assistants API. Response times are long (when you get a response at all), and token usage is high without the ability to choose what or when to truncate. That said, it’s still in beta, so we probably shouldn’t expect rock solid performance.

But it’s possible problem 3 could be aided by adjusting your prompt. Could you share the relevant part of your prompt? Someone here might be able to help you improve your success rate.

1 Like
4

Thank you for guiding me towards solving problem 3. Now that I know, I will experiment with tuning the prompt.

1 Like