So, I have put together my own text to SQL query system. It only uses two tables, and is restricted to generating analytics about my query log activity. It was surprisingly easy to do. Here is the development conversation: https://chat.openai.com/share/411300eb-bb09-4b96-ac4b-f167eb39b03b
As for bad SQL, this is my process:
User submits text request → Request is submitted to model along with message which contains restrictions → Model evaluates and returns SQL statement or error message → if SQL statement returned, that SQL is submitted to model again for double-verification against restrictions → If SQL statement is safe, then it is processed and output returned as html to user.
In addition to the model checks, my code also checks for forbidden statements and only submits SQL to read-only database.
Using gpt-3.5-turbo-16k. No one is more surprised than me at how well it works.