I am looking to incorporate GPT- 4 into my workflow to analyze and process data from various sources. The data I have can be in various formats, including CSV and databases like SQL. Could you provide any guidance on the most efficient way to prepare and present this data to GPT-4 for optimal processing? I’m particularly interested in ensuring that the data is concise and reduces token usage, given the token limits of the model.
One method to make a pretrain is convert the return of Query of SQL database to a JSON and inject at the start of the msg, but this going to be expensive if yu have a big database.
Can you try to check if Azure can conect directly to the database, l read something on doc about that one day-