I am blown away by the difference the token amount makes. I am currently using the 8k model, which is almost 2x the size of the gpt-3 model and it makes a huge difference. Token compression aside, the increase in token amount sees an exponential increase in the power of this api. I believe the token limit’s effect on the api is akin to the effect that memory had on the early computer.
If you have any questions on how the new model performs, please ask and I will be happy to answer!
the grammar is ok, spelling punctuation too
it seems like it doesnt understand the request, it returns part of the prompt in arabic for example instead of executing the instruction, strange things of that kind.
but maybe i should adapt the way the prompts are built,
will test further…
Moved from a separate thread
I’ve been trying to get GPT 3.5 turbo to return JSON, it succeeded most of the time, but I had to clean up the output behind it, found extra properties not mentioned in system message, and insist on returning empty fields when not relevant.
Just changed model name to GPT4 today, it is perfect all the time, follows instructions to the inch, gives exactly what I asked, no more, no less!!
We are seriously considering using it in one of the products, possibilities are limitless, basically you can develop an API by merely explaining what you want in natural language with perfectly structured response that you can expose as API with reasonable confidence that you won’t be let down.
Future is EXCITING!
It is still infeasible economically for most applications though. Is the cost coming down soon?
Sadly Arabic and knowledge, Arabic culture and Islam for example is extremely subpar, all model including GPT4 keep hallucinating and returning non sense, if I had a bit more time I would have created an eval, it is a very good subject to do an eval on.