1

image
image962×617 24.3 KB

There are 75 characters in “The Best Soft Boiled Eggs Recipe: Tips and Tricks for Perfectly Cooked Eggs”, but ChatGPT keeps insisting on other numbers. What’s going on?

1 Like
2

AI hallucination

1 Like
3

Do you know how I might be able to fix this? Is it possible to provide instructions to ChatGPT such that it figures out the right way to count characters?

4

The current generation of OpenAI GPTs do not count in the completion but the API does returns that info (as I recall OTTOMH).

Sorry to inform you @AiNewbie .

Note: I will confirm this for you when I am back at my desk.

5

I have asked AI to perform similar tasks via API and with succes. It may be, that ChatGPT is not so trained for this kind of stuff but more to provide humanlike responses.

6

Hi @AiNewbie

Just for you, I confirmed that the OpenAI completion API method returns the usage:

"usage":{
    "prompt_tokens":8,
    "completion_tokens":8,
    "total_tokens":16}
}

Completion Response:

{
"id":"cmpl-6m19S7dblahblah1CskqR8VRqJjZj6r",
"object":"text_completion",
"created":1676902190,
"model":"text-davinci-003",
"choices":[
    { "text":"\n\nMy favorite color is blue.",
      "index":0,
      "logprobs":null,
      "finish_reason":"stop"}
    ],
"usage":{
    "prompt_tokens":8,
    "completion_tokens":8,
    "total_tokens":16}
}

Of course, this is not “characters” it is “tokens” so probably not useful, sorry.

Hope this helps.

1 Like
7

I am facing the same issue. I tried in so many different ways to make it count characters and failed. Even created a separate chat with a prompt “Act as a word and character counter”. Failed. Failed. Failed.

1 Like
8

There’s same issue for Chinese characters, I notice when I asking for a nickname

9

image
image1170×1296 131 KB

I’m having a similar problem, I’m basically asking it to count letters in a row and it’s off by a few letters. One time i asked it how many words it just typed and it said “46.” It was 107.

10

Asking for a numbered list of the characters counted can help. However it sometimes mess it up by counting a word two times, and it will sometimes conclude with a different number then the list shows.

11

ChatGPT can’t count, but it can easily learn to count. One just needs to use the right prompt. Here is my prompt that teaches the system to count.

Let’s play a counting game. I say a word and a letter. You count the occurrences of the letter in the word. I say add:d. You say a-0 d-1 d-2. There are two occurrences of d in add. I say sincere:e. You say s-0 i-0 n-0 c-0 e-1 r-1 e-2. There are two occurrences of s in sencere. Ready to play?

1 Like
12

Using your method, I was able to count the number of characters! However, it takes a lot of time and tokens.

Instruction: Answer the number of words in the sentences I give you.
Example:
Q: I saw you on the bridge.
A: I-1 -2 s-3 a-4 w-5 -6 y-7 o-8 u-9 -10 o-11 n-12 -13 t-14 h-15 e-16 -17 b-18 r-19 i-20 d-21 g-22 e-23 .-24 The answer is 24
Q: Why is blue the sky?
A: W-1 h-2 y-3 -4 i-5 s-6 -7 b-8 l-9 u-10 e-11 -12 t-13 h-14 e-15 -16 s-17 k-18 y-19 ?-20 The answer is 20
Are you ready?

13

I have improved my method. My method allows ChatGPT3.5 to count number of words in a sentence, types of words in a sentence, multiple letters in a sequence of letters, occurrences of a number in the first few digits of pi, etc.

The approach is to divide the task into three steps: dividing elements of a given input into an ordered sequence (listing), evaluating each element in the sequence, and keeping a tally. I believe these skills (listing, evaluating, and keeping a tally) are the more basic skills from which counting is built.

Below is my prompt for preparing ChatGPT to count.

Let’s play a counting game. I will give you a search term and a search set. You tell me how often the search term occurs in the search set. Divide the search set into an ordered list of items. Move through the list evaluating each item. Does the item under consideration match the search term? Keep a tally, starting with zero. If there is a match, increment the tally by one. Stop at with the last item in the search set.

Use a markdown table to show your work. Have a column for Row #, Item, Evaluation, and Tally.

For example: I say: How often does “m” occur in “mommy”? You say:

Row # - Item - Evaluation - Tally
1 - m- Yes - 1
2 - o - No - 1
3 - m - Yes - 2
4 - m - Yes - 3
5 - y- No - 3

“m” occurs three times in “mommy”.

Here is a second example. This one has a search term with two letters. I say: la-lalaala. You say:

Row # - Item - Evaluation - Tally
1 - la - Yes- 1
2 - al - No- 1
3 - la - Yes - 2
4 - aa - No - 2
5 - al - No - 2
6 - la - Yes - 3

“la” occurs three times in “lalaala”.

Here is a third example. I say: “Count odd numbers in the sequence 3, 50, 9”. You say:

Row # - Item - Evaluation - Tally
1 - 3 - Yes- 1
2 - 50 - No- 1
3 - 9 - Yes - 2

There are two odd numbers in the sequence “3, 50, 9”.

Ready? How often does “p” occur in “puppet”?

14

The example provided in the original post was actually a simplified demonstration of real problem.

I was actually testing out ChatGPT’s ability to output meta titles and meta descriptions (for SEO purposes). For example, how do you tell ChatGPT, “write a meta description for article x that is 155-160 characters long”? In my testing, the results were always either under or over the given range.

After a few iterations of testings, I realized that ChatGPT can’t count characters correctly. So, instead of giving ChatGPT an example string to count like in the original example, maybe a better example would have been to ask ChatGPT to produce a string that’s within a range of characters.

15

Try it with a longer string, like 200-300 characters. It still messes up, but now I can see why. It’s actually changing the text before counting it.

Using cliff.Iandesman’s prompt above, I gave ChatGTP the following string to evaluate the number of characters (minus the quotes):

“This is a test paragraph for ChatGTP. It needs to count all of the characters in this set of text. It needs to include all characters, including punctuation and spaces. Do you think it can do this with an entire paragraph? It’s failed in the past, so this is a good test.”
This string has 271 characters (according to Microsoft Word).

But, when ChatGPT evaluated the string character by character, it gave me the following incorrect text:

“This is a test paragraph for ChatGTP. It needs to count all characters, including punctuation it’s a good this with a entire past, so this is a good test. It’s failed in this is a good test.”

However, it correctly identified that string as having 190 characters. So, technically it’s counting the text correctly, it’s just rewriting the text incorrectly before it counts it.

Here’s ChatGTP’s exact response:

Text: “This is a test paragraph for ChatGTP. It needs to count all of the characters in this set of text. It needs to include all characters, including punctuation and spaces. Do you think it can do this with an entire paragraph? It’s failed in the past, so this is a good test.”

Row # Item Evaluation Tally
1 T Yes 1
2 h Yes 2
3 i Yes 3
4 s Yes 4
5 space Yes 5
6 i Yes 6
7 s Yes 7
8 space Yes 8
9 a Yes 9
10 space Yes 10
11 t Yes 11
12 e Yes 12
13 s Yes 13
14 t Yes 14
15 space Yes 15
16 p Yes 16
17 a Yes 17
18 r Yes 18
19 a Yes 19
20 g Yes 20
21 r Yes 21
22 a Yes 22
23 p Yes 23
24 h Yes 24
25 space Yes 25
26 f Yes 26
27 o Yes 27
28 r Yes 28
29 space Yes 29
30 C Yes 30
31 h Yes 31
32 a Yes 32
33 t Yes 33
34 G Yes 34
35 T Yes 35
36 P Yes 36
37 . Yes 37
38 space Yes 38
39 I Yes 39
40 t Yes 40
41 space Yes 41
42 n Yes 42
43 e Yes 43
44 e Yes 44
45 d Yes 45
46 s Yes 46
47 space Yes 47
48 t Yes 48
49 o Yes 49
50 space Yes 50
51 c Yes 51
52 o Yes 52
53 u Yes 53
54 n Yes 54
55 t Yes 55
56 space Yes 56
57 a Yes 57
58 l Yes 58
59 l Yes 59
60 space Yes 60
61 c Yes 61
62 h Yes 62
63 a Yes 63
64 r Yes 64
65 a Yes 65
66 c Yes 66
67 t Yes 67
68 e Yes 68
69 r Yes 69
70 s Yes 70
71 , Yes 71
72 space Yes 72
73 i Yes 73
74 n Yes 74
75 c Yes 75
76 l Yes 76
77 u Yes 77
78 d Yes 78
79 i Yes 79
80 n Yes 80
81 g Yes 81
82 space Yes 82
83 p Yes 83
84 u Yes 84
85 n Yes 85
86 c Yes 86
87 t Yes 87
88 u Yes 88
89 a Yes 89
90 t Yes 90
91 t Yes 91
92 i Yes 92
93 o Yes 93
94 n Yes 94
95 space Yes 95
96 i Yes 96
97 t Yes 97
98 s Yes 98
99 space Yes 99
100 a Yes 100
101 space Yes 101
102 g Yes 102
103 o Yes 103
104 o Yes 104
105 d Yes 105
106 space Yes 106
107 t Yes 107
108 h Yes 108
109 i Yes 109
110 s Yes 110
111 space Yes 111
112 w Yes 112
113 i Yes 113
114 t Yes 114
115 h Yes 115
116 space Yes 116
117 a Yes 117
118 space Yes 118
119 e Yes 119
120 n Yes 120
121 t Yes 121
122 i Yes 122
123 r Yes 123
124 e Yes 124
125 space Yes 125
126 p Yes 126
127 a Yes 127
128 s Yes 128
129 t Yes 129
130 , Yes 130
131 space Yes 131
132 s Yes 132
133 o Yes 133
134 space Yes 134
135 t Yes 135
136 h Yes 136
137 i Yes 137
138 s Yes 138
139 space Yes 139
140 i Yes 140
141 s Yes 141
142 space Yes 142
143 a Yes 143
144 space Yes 144
145 g Yes 145
146 o Yes 145
147 o Yes 145
148 d Yes 145
149 space Yes 145
150 t Yes 145
151 e Yes 145
152 s Yes 145
153 t Yes 145
154 . Yes 145
155 space Yes 145
156 I Yes 145
157 t Yes 145
158 ’ Yes 145
159 s Yes 145
160 space Yes 145
161 f Yes 145
162 a Yes 145
163 i Yes 145
164 l Yes 145
165 e Yes 145
166 d Yes 145
167 space Yes 145
168 i Yes 145
169 n Yes 145
170 space Yes 145
171 t Yes 145
172 h Yes 145
173 i Yes 145
174 s Yes 145
175 space Yes 145
176 i Yes 145
177 s Yes 145
178 space Yes 145
179 a Yes 145
180 space Yes 145
181 g Yes 145
182 o Yes 145
183 o Yes 145
184 d Yes 145
185 space Yes 145
186 t Yes 145
187 e Yes 145
188 s Yes 145
189 t Yes 145
190 . Yes 145

The total number of characters in the provided paragraph, including punctuation and spaces, is 190. If you have any more questions or requests, please feel free to ask!