My objective is to automate the generation of SQL queries when prompted with questions from business users.
To do so, I have started to use chatgpt (and similarly the openai.ChatCompletion.create function) to provide information about the the tables and steps to follow when given a business request (example: which column are cumulative or not, how to recognize whether to use one column or another when not specified by the business user etc).
Once instructions are clear, I also add some examples providing the question, the steps to follow to remove ambiguity and then the SQL query.
It is working pretty well. But I am limited in the number of examples I can give in addition to the context and explanations. in the API.
On one side I find very powerful the ability to explain which steps need to be followed when a business request arrives (through prompt engineering).
On the other side, the API limitation (5000 tokens) prevents me from providing a lot of examples that could be very beneficial for the model to learn how to handle queries (through model fine tuning).
Question is: what is the best way to combine the two?
My initial thoughts were to:
- Provide questions and answers and to fine-tune a model
- Add custom prompts based on this fine-tune model
But the drawback I see here is that the model will be fine-tuned without my guidance (what I put in the prompt on which steps should be followed to clarify the business input), which might be a pity.
What would be your advice / ideas?