i need to implement chatgpt into our software, using it as an assistant who takes a request, transform it to a query (using a table schema).
I tried Assistant on the playground, giving it a tableschema file (which contains just the table data like columns and info about fields) and making the system message as detailed as possible (like "using the tableschema file, generate an sql query ecc…). It gives me a query that i execute on SQL and it is pretty good.
My problem is that costs are HUGE because of the all context.
I don’t know if Assistant is the right choice, i don’t need any context saved. I just need chatgpt to read the structure of the database e create a query, that’s all.
What do you suggest? I didn’t find anything about giving chatgpt a file through API (the only way is assistant api). Do you think the cheapest way is to use classic chat completation with a good system message? Including the table schema inside the text? Can you suggest something?
I’d avoid assistants like the plague, although I can see their value for some use cases.
How large is your schema file? Retrieval is just gonna turn it into text anyways. running your own vectordb is always an option if your schema is gigantic.
Thanks for answering
Schema is actually simple, some tables, some fields, i’m not planning to include all tables and fields but just the one i need.
How would you use a vectordb in this case?
I don’t understand how this could be so expensive, unless you are uploading the file(s) as knowledge.
Perhaps I’m not understanding your issue, but I thought that was what you were doing. No need to upload your data to Assistants. You only need:
An API to access your data. If it is spreadsheet, for example, you could upload it to Google Sheets and use their API to access.
An understanding of how to query your data through the API.
A system prompt that explains how to query the API to the LLM.
With this, you could actually create a GPT that connects to your API as an Action for knowledge. So now, the LLM would simply translate the user’s query into the query language and execute that via the API.
That’s could be good but can custom gpt be access by API? I need it to implement into a custom software so I need everything to be doable via api. Thanks for response:)
If you are looking to save in cost, I would think it’s more like:
GPT -----> Your API
User asks question about your data. GPT looks at your database schema and creates query language and sends that query to your API. Your API processes the request and sends back the appropriate data. GPT looks at the data and formulates a response for the user.
All this without having to upload your entire database to the GPT.
Ok got it.
I have not so many tables/fields… so do you think that having it in the prompt is a good idea? Or is there a method to use files? I can’t think of any.
Just try to put it into the prompt. A couple of cents extra spent on prompts is probably gonna be cheaper than dozens of hours setting up and maintaining all the ancillary crap.
And even if you do decide that it’s too expensive, you can always opt for that later.
How would you organise the prompt?
I’m sorry for all that question, I want to learn as much as possible, I tried with a speech-style prompt but it seems so long. It is normal to have a long prompt?
Do you use function calls at all at the moment? Leaving costs and any other hard feelings about Assistants aside for a moment, unless I am getting this wrong you can achieve this with a clearly defined function call (incl. a detailed function description that sets out your SQL request parameters) plus sufficiently clear and detailed instructions so the Assistant understands how to turn user messages into SQL queries which are then executed in the background via an API connection to your database.
If you are looking for exact data points from your SQL table(s), then I would stick to using SQL queries.
