ChatGPT Can't Count Characters?

There are 75 characters in “The Best Soft Boiled Eggs Recipe: Tips and Tricks for Perfectly Cooked Eggs”, but ChatGPT keeps insisting on other numbers. What’s going on?

1 Like

AI hallucination

1 Like

Do you know how I might be able to fix this? Is it possible to provide instructions to ChatGPT such that it figures out the right way to count characters?

The current generation of OpenAI GPTs do not count in the completion but the API does returns that info (as I recall OTTOMH).

Sorry to inform you @AiNewbie .

Note: I will confirm this for you when I am back at my desk.

I have asked AI to perform similar tasks via API and with succes. It may be, that ChatGPT is not so trained for this kind of stuff but more to provide humanlike responses.

Hi @AiNewbie

Just for you, I confirmed that the OpenAI completion API method returns the usage:

"usage":{
    "prompt_tokens":8,
    "completion_tokens":8,
    "total_tokens":16}
}

Completion Response:

{
"id":"cmpl-6m19S7dblahblah1CskqR8VRqJjZj6r",
"object":"text_completion",
"created":1676902190,
"model":"text-davinci-003",
"choices":[
    { "text":"\n\nMy favorite color is blue.",
      "index":0,
      "logprobs":null,
      "finish_reason":"stop"}
    ],
"usage":{
    "prompt_tokens":8,
    "completion_tokens":8,
    "total_tokens":16}
}

Of course, this is not “characters” it is “tokens” so probably not useful, sorry.

Hope this helps.

1 Like

I am facing the same issue. I tried in so many different ways to make it count characters and failed. Even created a separate chat with a prompt “Act as a word and character counter”. Failed. Failed. Failed.

1 Like

There’s same issue for Chinese characters, I notice when I asking for a nickname

I’m having a similar problem, I’m basically asking it to count letters in a row and it’s off by a few letters. One time i asked it how many words it just typed and it said “46.” It was 107.

Asking for a numbered list of the characters counted can help. However it sometimes mess it up by counting a word two times, and it will sometimes conclude with a different number then the list shows.

ChatGPT can’t count, but it can easily learn to count. One just needs to use the right prompt. Here is my prompt that teaches the system to count.

Let’s play a counting game. I say a word and a letter. You count the occurrences of the letter in the word. I say add:d. You say a-0 d-1 d-2. There are two occurrences of d in add. I say sincere:e. You say s-0 i-0 n-0 c-0 e-1 r-1 e-2. There are two occurrences of s in sencere. Ready to play?

1 Like

Using your method, I was able to count the number of characters! However, it takes a lot of time and tokens.

Instruction: Answer the number of words in the sentences I give you.
Example:
Q: I saw you on the bridge.
A: I-1 -2 s-3 a-4 w-5 -6 y-7 o-8 u-9 -10 o-11 n-12 -13 t-14 h-15 e-16 -17 b-18 r-19 i-20 d-21 g-22 e-23 .-24 The answer is 24
Q: Why is blue the sky?
A: W-1 h-2 y-3 -4 i-5 s-6 -7 b-8 l-9 u-10 e-11 -12 t-13 h-14 e-15 -16 s-17 k-18 y-19 ?-20 The answer is 20
Are you ready?

I have improved my method. My method allows ChatGPT3.5 to count number of words in a sentence, types of words in a sentence, multiple letters in a sequence of letters, occurrences of a number in the first few digits of pi, etc.

The approach is to divide the task into three steps: dividing elements of a given input into an ordered sequence (listing), evaluating each element in the sequence, and keeping a tally. I believe these skills (listing, evaluating, and keeping a tally) are the more basic skills from which counting is built.

Below is my prompt for preparing ChatGPT to count.


Let’s play a counting game. I will give you a search term and a search set. You tell me how often the search term occurs in the search set. Divide the search set into an ordered list of items. Move through the list evaluating each item. Does the item under consideration match the search term? Keep a tally, starting with zero. If there is a match, increment the tally by one. Stop at with the last item in the search set.

Use a markdown table to show your work. Have a column for Row #, Item, Evaluation, and Tally.

For example: I say: How often does “m” occur in “mommy”? You say:

Row # - Item - Evaluation - Tally
1 - m- Yes - 1
2 - o - No - 1
3 - m - Yes - 2
4 - m - Yes - 3
5 - y- No - 3

“m” occurs three times in “mommy”.

Here is a second example. This one has a search term with two letters. I say: la-lalaala. You say:

Row # - Item - Evaluation - Tally
1 - la - Yes- 1
2 - al - No- 1
3 - la - Yes - 2
4 - aa - No - 2
5 - al - No - 2
6 - la - Yes - 3

“la” occurs three times in “lalaala”.

Here is a third example. I say: “Count odd numbers in the sequence 3, 50, 9”. You say:

Row # - Item - Evaluation - Tally
1 - 3 - Yes- 1
2 - 50 - No- 1
3 - 9 - Yes - 2

There are two odd numbers in the sequence “3, 50, 9”.

Ready? How often does “p” occur in “puppet”?

The example provided in the original post was actually a simplified demonstration of real problem.

I was actually testing out ChatGPT’s ability to output meta titles and meta descriptions (for SEO purposes). For example, how do you tell ChatGPT, “write a meta description for article x that is 155-160 characters long”? In my testing, the results were always either under or over the given range.

After a few iterations of testings, I realized that ChatGPT can’t count characters correctly. So, instead of giving ChatGPT an example string to count like in the original example, maybe a better example would have been to ask ChatGPT to produce a string that’s within a range of characters.