I’m developing an application to manage customer complaints via a CSV file, but I’ve encountered limitations with the code interpreter regarding semantic similarity. I’m considering converting the complaints into a PDF file and then sending them to an assistant using ChatGPT’s API. However, I’m unsure whether ChatGPT’s information retrieval tool can analyze entire documents or only retrieves the most similar texts for response.
For example, if I ask ChatGPT to identify the top 3 most common types of complaints from a document containing 10,000 complaints, does it have the capability to analyze the entire document, classify all complaints, and then respond based on those complaints rather than relying solely on text similarity? From my understanding about RAG, ChatGPT’s retrieval selects top documents based on semantic similarity, which leads me to believe this may not be feasible.
If not feasible, would it be practical to have the number of ‘N’ complaints within the allowed context?.
I don’t know the answer to this myself, but I’ll help refine the question, for everyone else including me, for an OpenAI expert to answer, if someone knows, which is…Is it normally practical to run the same prompt/question on multiple data inputs in the same prompt?
For example, if I have 1000 paragraphs, and I need to run a “Please summarize this paragraph in a single sentence” query on all 1000 paragraphs, is thing going to even work well? Or will there always be some “attention bleeding” across paragraphs where the context of one paragraph is “considered” by some other paragraph which would corrupt the accuracy, compared to what you’d get with 1000 separate inferences.
I’m just adding to this thread, because this is a very important issue/question lots of LLM users are going to have to grapple with in real world situations like this customer complaints example.
In other words I have no doubt you’d get better results by running 1000 separate inferences, unless OpenAI has implemented some sort of parallel batching mechanism where you can submit the same query across an array of different inputs. Potentially even a special text delimiter could be used to do this, which would mean there’s no API changes required but where people can run batches of queries without the overhead of 1000s of HTTP calls.
The short answer to this is that you are unlikely to get a response from ChatGPT that relies on systematic analysis that a human would perform in order to arrive at the conclusion of what the top 3 most common complaints types are.
It’s been nearly a year since I first tested something like this based on a larger data set - at the time with the original gpt-4 and not using the assistant. My experience at the time was that while you get a “reasonable answer”, it is not a good way to extract systematic patterns.
In order to achieve a more accurate (pattern) analysis, the best way would still be to break down the process into individual steps, from classification, pattern analysis to aggregation into insights.