Is it really this bad?
Today I needed to create a new assistant. It featured a single file that contained an albeit large json with 500 entries, but each has 2 keys, “segment_name”, “ID”.
I had a simple prompt which was basically, "take this business description and look through the file and match it to a segment that relates to it. Return me the segment_name and ID.
9 times out of 10 it would be incorrect, seemingly to just pick any segment_name, and then just make up an ID which wasn’t even related. And the ID’s are just 1 to 500.
As an example, one description I gave it was “We manufacture car parts”, and even though there is a segement_name along the lines of “manufacturer of automotive parts” it came back with “private security” or something like that with an ID that wasn’t even from that element.
The more I use 4.1 in general the more I feel like switching to another LLM provider - but for now I’d like advice on whether an older model is better for this use case? Or is it time to just ditch it?