I asked chatgpt to return lists of words with a specific number of letters. I also indicated which position within the word a certain letter should take but he didn’t understand. Returns longer or shorter words. It is a big problem in my opinion.
ChatGPT—and all GPT models for that matter don’t know what words or letters are in any real sense, they communicate in tokens.
The models which power ChatGPT have a vocabulary of 100,256 tokens (you can see them all here: https://github.com/niieani/gpt-tokenizer/blob/main/data/cl100k_base.tiktoken). Those are the smallest possible units ChatGPT can understand, so asking ChatGPT to count letters is setting it up to fail. So when it does fail it’s not really the model’s fault.
Now, you’re not wrong for having thought this would be something it could do, it seems incredibly elementary compared to some of the things it can do, but there is the plain and simple fact of it, this is a task which is beyond its capabilities.
Thanks for your answer. In your opinion It Will be possible to integrate a tool in order to solve this problem? I think it Is really a big problem that AI cannot return words with specific indications.
Yes, it would be possible to integrate a tool for words, just as with math. A ChatGPT plugin could be developed with functions like “count words, count tokens, count characters, parse strings, play hangman” etc.
What would be the motivation for a developer to spend $240 a year on plus, to set up their own domain and API backend to service user requests, and write such a plugin, just to improve OpenAI’s service for free?
Or you can tell ChatGPT with advanced data analysis enabled that it cannot do this itself and it must use python to answer the task.