Hello - is it anyhow possible to send a file to the api (eg. a txt-file or also an xlsx-file) and get some answers for this files?
So some answers for the text in the file for example or some question about the excel-sheet?
1 Like
- look into “embedding” that’s the complicated way
- or use something like TIKA to extract text from any document
- or, simplest way, is get text from the file yourself (however you want)
If doing 2 or 3, above then send text from any method above into the prompt, and say
“You will answer questions about the following text: {text}. {question_to_ask}”
Interested in this as well. The former /answers endpoint which is now deprecated is the sort of flow I’d like to have.
My document/text is a large JSON file which doesn’t seem to make sense to send with every prompt (occupies tokens, and does not change often).
Is there any workarounds apart from fine-tuning every question/answer?
Currently you do have to send the entire “context” to the AI for every question. It’s not an OpenAI limitation, but a limitation of LLMs themselves.
Luckily everyone has this same “problem” and so there is a huge number of researchers working to make the context window able to be longer and at less cost.
Thanks, good to know.
How would embeddings work? Would i need to embed a document of Q&A, and then match the prompt with the most similar Q&A pair?
I’m referencing cohere’s explanation of Semantic Search: Semantic Search
Embeddings work by matching a query in a text, and filtering it by similarity (although there are many other filtering options). This may be clearer with an example:
Imagine you have a txt about the climate change and you mention the reasons and impact.
If you query that text via embeddings and want to get the reasons, the embedding model (probably ada-002) will return the chunk of text that has a more high probability of being the answer to that query.
This is a small outline of what the process for QA on documents could be like:
Note: this is a really superficial explanation about embeddings. In reality, you’d need to learn about what a vector is, vector stores, databases, etc.
or you could just steal some code from the internet, less ethical but does the job
Here’s a good intro to embeddings video:
Basically you use a VectorDb of all your pre-generated embeddings for all chunks of your reference data, and then use your “user search” string to find the top closest matches and then extract those matching sections of your document chunks and feed it to the LLM as “context” for the question being asked.
So the embeddings are just used for the cosine search, in order to build up the most compact “context” you can that comes from the most likely text to contain enough knowledge for the LLM to use to build an answer from.
Embeddings appears to be a way to find similar items to a textual query inside a any text, i.e. similarity based text-search. This is actually not what people are looking for.
Open AI’s ChatGPT interface itself allows to actually submit a file, grasp and reason over the file. How is this possible via the API?
2 Likes
Im unaware of an API call capable of this. Instead, youll have to make a VDB, parse your doc and chunk it into sections, throw the sections into the VDB, then use the user query to retreive the chunk of data with which the AI generates a response.
Sounds like a lot, but its not really. But the end prompt ends up being a user query with relevant data sent together to the API.
Imagine you have a large amount of Excel data and the AI is supposed to reason over the whole dataset (e.g. average of Score in combination with other fields). How is a VDB and similarity-based text extraction supposed to help in this case?
The AI is required to comprehend and derive data from the Excel sheet as a whole.
Since you are using the API, you might want to consider the assistant API.
You will have to:
- Create an assistant
- Upload the files to the assistant
- Attach the files to the assistant
- Perform your query as per the docs.
1 Like