ChatGPT cannot count words or produce word-count-limited text

ChatGPT
1

My question to ChatGPT - “Write an essay summarising the plot of Shakespear’s “Twelth Night” in 500 words. At the end of the essay, state how many words it contains.”

Last line of the resulting (excellent) essay - “This essay has 538 words.”

Actual number of words according to Libre Office - 355. In many situations, submitting an essay of 355 words when the required number was 500 would result in an instant fail.

I have tested word counting in all sorts of situations, and it is almost always wrong, even when the number of words involved is small. “Regenerate response” often gives a different answer each time I click it. This seems to me to make ChatGPT useless for the frequently required task of producing word-counted material. Or am I doing something wrong?

Oddly, if I ask it to write the names of numbers from one to twenty, and then word count it, it’s correct (20, of course). If I paste those 20 words in twice, it correctly counts 40 words. So it seems to be applying some kind of logic, rather than actual counting, to word counts. Weird.

4 Likes
2

ChatGPT is not a calculator.

ChatGPT is a chatbot based on language prediction. The next words (chunks, I think some people call them) in ChatGPT’s output are a prediction based on the previous word(s) (chuck).

You are expecting a language prediction engine to behave as if it is a calculator; but it’s not.

Yes, the “logic” is the “logic” of a GPT-based LLM, not a calculator.

Not “weird” when we understand what ChatGPT is and is not.

HTH

1 Like
3

This does not make a lot of sense… Counting is not math. Counting is counting. He even said that if he pastes 20 words in, ChatGPT can count them. You’re just making stuff up, confidently, just like ChatGPT does! Lol

Also, you can put actual math problems into ChatGPT and it will calculate them all out for you.

Ops issue is that it cannot keep track of what it is saying properly. I also run into this with coding. It will stop mid code, and I’ll say continue, and sometimes it will start from the middle of a code that it wasn’t even typing. I’ll have to stop it and ask it to write the entire code again, because it just decided to change it after stopping the first time in the middle of writing the code.

So, yes, it’s weird! Even if you actually know how ChatGPT works.

7 Likes
4

Counting is math.

1 + 1 + 1 = 3  is math

Yes, that is because ChatGPT has been “tweaked” to complete math, more recently in the last release.

No, I simply have written a lot of code using the OpenAI API, used ChatGPT extensive to help in very tasks, and have read many papers on how GPT models are architectures work. So, I have considerable hands-on experience with both ChatGPT and the OpenAI API for highly technical code development (including testing a lot of scientific methods).

The “G” in GPT mean “generative”.

GPT engines are “Generative” in nature. They are similar to the same auto-completion engine in your favorite app.

GPT engines are not calculators. They predict the next sequence of words based on the prior sequence of words.

That’s what they do.

2 Likes
5

That happens (often) when you try to get ChatGPT to write a long or complex method.

ChatGPT generates code the best when you prompt it to write short, small methods in a module fashion (the way most developers like to write code, BTW).

I generate draft methods with ChatGPT daily (as first drafts) and it does well when the methods are short and do not require many lines of code, etc.

9

OK… well explain this, if it can’t count. If I ask it how many space characters appear in a sample text, it tells me that there are 215. Word processing app tells me that there are 214 words. So that’s how you get it to count words. Count spaces instead. (If I ask ChatGPT how many words are in that text, it says 325 - wrong!).

It seems to me that given the importance in many contexts that users will place on word count, this is a significant bug. Or “functional limitation” if you don’t like the word “bug”.

[Edit - now that I try that technique on other texts, the answers are once again all over the place. I think I give up…]

1 Like
10

Have you tried to have ChatGPT count each word as it writes? Something like “number each word as you write it like this: How 1 now 2 brown 3 cow 4 ?” Then all you would have to do is delete the numbers 0-9.

11

Or …,

  1. Use the OpenAI API and easily count the words when the completion returns.

  2. Wait for the ChatGPT API and count the words when the completion returns.

There is only “so much” you can do with the web app, so better to wait for the “coming soon” ChatGPT API and count the words yourself :slight_smile:

12

Guytron - thanks - I guess I’m trying to imagine I’m a kid writing a school essay - often given a word count to achieve with a margin of error of perhaps 10%. Or a Uni essay. This after all is what seems to be terrifying the education world at the moment! But when it comes down to it, given that there’s no simple way to control word count, or no reliable way, I’m getting the feeling that academia need not worry too much. Of course if you really know what you are doing you could get ChatGPT to write an essay summary first, then get it to add detail to each item in the summary one by one, then manually edit for length (and accuracy!), and combine using WP software. But some students would probably decide it was easier to write the essay themselves - which might be a good thing.

ruby_coder - thanks - I guess I’m expecting too much too soon. I read a piece online about how ChatGPT generates texts of various sizes taking stuff into consideration which actually went over my head (not hard to achieve) and I can see this is way more complicated than I realise. I think I’m encountering the gap between the online / news media hype, and the current reality (which none the less impresses me hugely).

13

By the way, when studying this I’ve just come across a handy online tool for checking word counts (and character counts and other stuff) - https://wordcount.com/

14

I think you and many others are looking at a beta marketing application called “ChatGPT” and you are trying to take this beta marketing application and program it like it was an API.

The API is called OpenAI API and requires a lot of programming to begin to “behave” like the ChatGPT web application.

Because of high demand for ChatGPT, OpenAI (and Microsoft) have already announced a ChatGPT API.

So, when the ChatGPT API arrives, programming ChatGPT-based applications will be a lot easier (we all think).

In the meantime, enjoy ChatCPT for “what is is” and not “what it is not”. ChatGPT is a beta marketing application to demonstrate the capabilities of OpenAI. ChatGPT is based on a large language model, it’s not a calculator. If you want to count words today, you should use the OpenAI API, because then you can easily count words when the completion returns.

Better, as mentioned, wait for the “coming very soon” ChatGPT API, and count those words when completions are returned, because for most people who love ChatGPT, it’s better to wait for the ChatGPT API to be released instead of getting frustrated that the OpenAI API is not ChatGPT :slight_smile:

Hope this helps.

15

Thank you. Indeed, there’s so much that it does do very well that it’s kind of mean to pick on the things where there are - currently - perceived shortcomings. Don’t get me wrong - I am a huge fan. There’s a lot of negative press about it but it seems to me that it’s here, like it or not, and the thing to do is to get to know what it can do and how, both for oneself and for others, and use it to expand one’s own education, not replace it.

16

image
image1141×644 71.5 KB

ChatGPT cannot accurately count words and is not suitable for producing word-counted material. This is due to the fact that ChatGPT is based on language prediction and not a calculator. The best way to get ChatGPT to generate code is to prompt it to write short, small methods in a module fashion. Furthermore, ChatGPT can count space characters rather than words when asked to calculate a word count, but the results are often inaccurate.

Post summary using GPT-3

My question to ChatGPT - “Write an essay summarising the plot of Shakespear’s “Twelth Night” in 500 words. At the end of the essay, state how many words it contains.”

Last line of the resulting (excellent) essay - “This essay has 538 words.”

Actual number of words according to Libre Office - 355. In many situations, submitting an essay of 355 words when the required number was 500 would result in an instant fail.

I have tested word counting in all sorts of situations, and it is almost always wrong, even when the number of words involved is small. “Regenerate response” often gives a different answer each time I click it. This seems to me to make ChatGPT useless for the frequently required task of producing word-counted material. Or am I doing something wrong?

Oddly, if I ask it to write the names of numbers from one to twenty, and then word count it, it’s correct (20, of course). If I paste those 20 words in twice, it correctly counts 40 words. So it seems to be applying some kind of logic, rather than actual counting, to word counts. Weird.

1 Reply

Reply

created
1d
last reply
1d
11
replies
142
views
4
users
55

ruby_coder
1d
So it seems to be applying some kind of logic, rather than actual counting, to word counts. Weird.

ChatGPT is not a calculator.

ChatGPT is a chatbot based on language prediction. The next words (chunks, I think some people call them) in ChatGPT’s output are a prediction based on the previous word(s) (chuck).

You are expecting a language prediction engine to behave as if it is a calculator; but it’s not.

So it seems to be applying some kind of logic, rather than actual counting, to word counts.

Yes, the “logic” is the “logic” of a GPT-based LLM, not a calculator.

Weird.

Not “weird” when we understand what ChatGPT is and is not.

HTH

Reply

KingTabens
1d
This does not make a lot of sense… Counting is not math. Counting is counting. He even said that if he pastes 20 words in, ChatGPT can count them. You’re just making stuff up, confidently, just like ChatGPT does! Lol

Also, you can put actual math problems into ChatGPT and it will calculate them all out for you.

Ops issue is that it cannot keep track of what it is saying properly. I also run into this with coding. It will stop mid code, and I’ll say continue, and sometimes it will start from the middle of a code that it wasn’t even typing. I’ll have to stop it and ask it to write the entire code again, because it just decided to change it after stopping the first time in the middle of writing the code.

So, yes, it’s weird! Even if you actually know how ChatGPT works.

2 Replies

Reply

ruby_coder

1
1d
Counting is not math. Counting is counting

Counting is math.

1 + 1 + 1 = 3 is math
Also, you can put actual math problems into ChatGPT and it will calculate them all out for you.

Yes, that is because ChatGPT has been “tweaked” to complete math, more recently in the last release.

You’re just making stuff up, confidently, just like ChatGPT does! Lol

No, I simply have written a lot of code using the OpenAI API, used ChatGPT extensive to help in very tasks, and have read many papers on how GPT models are architectures work. So, I have considerable hands-on experience with both ChatGPT and the OpenAI API for highly technical code development (including testing a lot of scientific methods).

The “G” in GPT mean “generative”.

GPT engines are “Generative” in nature. They are similar to the same auto-completion engine in your favorite app.

GPT engines are not calculators. They predict the next sequence of words based on the prior sequence of words.

That’s what they do.

Reply

ruby_coder
1d
. I also run into this with coding. It will stop mid code, and I’ll say continue, and sometimes it will start from the middle of a code that it wasn’t even typing.

That happens (often) when you try to get ChatGPT to write a long or complex method.

ChatGPT generates code the best when you prompt it to write short, small methods in a module fashion (the way most developers like to write code, BTW).

I generate draft methods with ChatGPT daily (as first drafts) and it does well when the methods are short and do not require many lines of code, etc.

Reply

Ozpeter

1
1d
OK… well explain this, if it can’t count. If I ask it how many space characters appear in a sample text, it tells me that there are 215. Word processing app tells me that there are 214 words. So that’s how you get it to count words. Count spaces instead. (If I ask ChatGPT how many words are in that text, it says 325 - wrong!).

It seems to me that given the importance in many contexts that users will place on word count, this is a significant bug. Or “functional limitation” if you don’t like the word “bug”.

[Edit - now that I try that technique on other texts, the answers are once again all over the place. I think I give up…]

Reply

TL:DR;

ChatGPT cannot accurately count words and is not suitable for producing word-counted material. This is due to the fact that ChatGPT is based on language prediction and not a calculator. The best way to get ChatGPT to generate code is to prompt it to write short, small methods in a module fashion. Furthermore, ChatGPT can count space characters rather than words when asked to calculate a word count, but the results are often inaccurate.

GPT-3 talking about white spaces and count space characters

Organize the result using sequencia topics and formal language to create a post reply. (English Language)

ChatGPT is not suitable for producing word-counted material due to the fact that it is based on language prediction and not a calculator. Word counting can often be inaccurate, which makes it difficult to use ChatGPT for this purpose. However, it is possible to get ChatGPT to count space characters instead of words when asked to calculate a word count, although the results are often inaccurate.

To get the best results from ChatGPT when generating code, it is recommended to prompt it to write short, small methods in a module fashion. This is because ChatGPT performs best when generating short sequences of words.

Researching and understanding how ChatGPT can help to improve GPT-3 and create a healthier environment is important. To do this, one should consider the following:

  1. Research how ChatGPT can be used to improve natural language processing.
  2. Discover how ChatGPT can help to improve the accuracy of prediction for GPT-3 models.
  3. Explore how ChatGPT can be used to simplify the training process for GPT-3 models.
  4. Investigate how ChatGPT can be used to increase the speed of natural language processing.
  5. Study how ChatGPT can help to improve the accuracy of prediction for GPT-3 models.
  6. Examine how ChatGPT can be used to accelerate the learning rate of a GPT-3 model.
  7. Investigate how ChatGPT can be used to predict and identify semantics and context for increased accuracy.
  8. Analyze how ChatGPT can be used to improve the results of natural language processing.
  9. Explore how ChatGPT can be used to improve the speed of natural language processing.
  10. Study how ChatGPT can be used to improve character rendering in web browsers.

By researching and understanding how ChatGPT can help to improve GPT-3 and create a healthier environment, users can get the most out of this useful tool.

JSON Summary:

{
“Words”: 685,
“Tokens”: 607,
“Whitespaces”: 310,
“Linebreaks”: 12,
“Characters”: 3488,
“Letters”: 2844,
“ContentGaps”: 0,
“Questions”: 5,
“Conclusion”: “WebBrowsers and Character Rendering”
}

1 Like
17

So it appears to me my answers are constantly being directed a sertin direction. All though it can wright a grate letter or poem. It can’t find the actual number for my business proposal. And no matter how I ask the ai absolutely refuses to give real population numbers. This i believe is because the programers involved are biest.

18

Yes, because it is a language model, not a calculator.

ChatGPT predicts text based on probabilities which are based on models based on data from most public data sources.

You have auto-completion in your vary email composers and editors, right? Do you try to do math when you are auto-completing text in email?

ChatGPT does not “refuse” anything. These GPT models do not have the ability to make those kinds of decisions, to “accept” or “refuse” to answer.

Anywho, @jd2470 your post is hard to read due to all the spelling errors. You can submit your post to ChatGPT and I’m sure the language model can correct your spellings before posting :). That is a very good use case for a language model :slight_smile:

1 Like
21

If I may make a observation—a “normal” and simple software program can count words.

Someone has given us a screwdriver and we seem upset that it doesn’t bang in those old nails.

1 Like
22

I don’t think that quite works. All over the internet people are asking ChatGPT to write material of all kinds (text material) and in a significant number of situations, there’s a requirement to achieve a certain word count. That abilitiy might not be inherent in the way that it works, but I would have thought it would be something that the developers could have programmed outside of the central mechanisms. It’s kind of essential, I would have thought. It’s like having a wonderful screwdriver in the form of a handle with removable bits, except it only comes with one flat bit, not with a crosspoint and all the others. Also, its inabiliity in this respect seems to be almost unknown - I see constant references even in YouTube videos advising people how to use it, where word counts are being specified but there’s no mention of its limitiations in that respect. But I still think it’s amazing. I just drafted an email using it, concerning a technical subject that I am very familiar with, but I’m not that good at explaining. ChatGPT wrote the explanation, and I did the job of checking that it was correct. Done in about a minute, instead of half an hour and badly!

23

ChatGPT absolutely can “count” the number of words. You have to use a COT type prompt.
example:
"Q: How many words does “My dog is a cat” have ?
A: 5
Q: How many words does “My name is Bob and I like icecream” have ?
A: 8
Q: How many words does [YOUR_TEXT] have ?
A: "

it will give you the correct answer. also it can calculate arithmetic operations. there is just a way to ask. I invite you to read: https://learnprompting.org/

24

image
image960×2079 147 KB

25

image
image960×2079 162 KB

4 Likes