Ideas to improve Assistant performance

API
, , , ,
1

Hello OpenAI community!

  • Objective: Improve overall OpenAI Assistant performance in a specific topic

I use the OpenAI assistant. Yet, I’m excited to find ways to improve its performance, I’m asking the community for ideas and suggestions to improve it.
Idea :bulb: : i’m working on a project that uses open ai assistant to convert natural language to SQL query to execute it in a specific and private database “text-to-sql”

  • Idea I’m thinking of:
    • working with function in tools and pass function for testing if the SQL query that generate can execute in specific databases, and in I can pass demo database that I’m working as file.

help me with:

  • Sharing your experiences with using the OpenAI Assistant.
  • Propose ideas for improving its performance.
  • Discuss any workarounds or tweaks you’ve discovered.
4 Likes
2

I use assistant to extract key points from job description and match them with requirements from similar job description of the same role. The end goal is to have a hub of job insights for people to prepare their applications/career plans. Some takeaways I learned from the process so far:

  • Using the same model, assistant can be a bit less accurate than chat completion because there is no temp setting. It is important to test the prompts thoroughly and ensure it works with edge cases to. I find it quite unstable with structured output.
  • Response latency is really a thing, especially during peak hours. It took me up to 2 mins for a response sometimes, and I am not even using code retrieval or function calls. Timeout is really an issue for me right now that I even consider moving out of assistant until it is more stable. Other times assistant works great.

For the two points above, I am currently using a combination of assistant and chat completions.

  • Assistant swallows up quite a lot of tokens as it keeps the context. I use separate assistants for specific tasks to keep the context separate, reduces confusion and inaccuracy, as well as to control the cost.
  • I tried using function calling before but it does not always work as expected (missing parameters). Few-shot prompting can improve the situation but not guaranteed. I use chat completions for that task now and find its JSON output setting much more stable than assistant.
2 Likes
3

Thank you for your positive feedback on the Assistant API.
Also, welcome to the OpenAI developer forum!

The idea of using SQL to address the drawback of the Assistant consuming a large number of tokens during repetitive retrieval processes seems very reasonable and constructive.

We welcome opinions from others as well.

1 Like
4

Thank you for getting back to me.

I’m currently experiencing difficulty comprehending the exact workings of the function and how to ensure its capability to verify the generated SQL query. In case the query is invalid, I would like to prompt the Language Model Assistant to generate an alternative one.

Could you assist me with some references or ideas?