I digitized all the reports of a company who deals within the oil industry these and other records go back to the 1990s. They include text and images in standard formats such as word excel powerpoint and PDF.
If I wanted to use OpenAI tools on this dataset alone, or possibly more usefully in combination with the global dataset, is this possible with the OpenAI tools api?
Also once I had a combined global and private dataset, is it such that the private data then gets included in the global data set and no longer becomes private?
Also there are many diagrams and images within the dataset are these of no use within GPT-3 or would there be a way to include them in a relevant query?
I tried giving it links to various types of images (the text-davinci-003 model in particular) as context, and asked it to answer questions about them.
Flow charts and other diagrams, paintings, photos etc.
It very rarely came back with anything even remotely correct. It does seem to try to parse images, but it is really, really terrible at making any sense of them.
I thought it might understand clear diagrams, and it does often parse text in images ok, but the surrounding context is practically never right, less than random likelihood that it gives you something useful.
However, give it a document, like word doc, excel, presentation, markdown text and all kinds of other common formats, and it does usually extract the information fairly reliably, I find.