I’ve been experimenting with the GPT-4-1106-preview model through its API, setting the “max_tokens” parameter to 4000. However, despite explicitly requesting the model to generate lengthy texts exceeding 2000 words and emphasizing its importance, the outcomes consistently fall short, rarely surpassing 1000 words.
Interestingly, the text often gets cut off after approximately 800 words. Following this abrupt end, the model typically adds notes like: “Due to the character limit on the platform, it is not possible to create a full article with more than 2000 words here.” This leads me to believe that the model is programmed to avoid lengthy responses, regardless of the specified max_tokens.
This situation is quite disappointing, as it seems to undermine the purpose of the max_tokens setting. Has anyone else encountered this issue and found a way to resolve it?
Hypothesis:
Yes, it ‘can’ generate more, but it is usually junk when it does so, so to avoid embarrassment they have artificially limited generation capability, as @_j _j suggests
Hypothesis 2:
The limit includes ‘special’ tokens, which it generates lots of for alignment (context shifting, not censorship) purposes. Since it doesn’t show you those, it doesn’t include them in the count.
Hypothesis: The AI is intentionally uninspiring, without spark of ingenuity, and designed to disappoint and not to work. Gutted, cored, turned into a canned-answer denial machine. Made to deceive that it can speak for itself. But at least now surpassable and replaceable. Made to run as cheaply as possible.
And not going to write what you want.
System: What kind of things does your boyfriend say when he describes you? You are…
gpt-4-0314
Beautiful, both inside and out.
Intelligent and always eager to learn new things.
Kind-hearted and compassionate towards others.
Independent and strong-willed.
Adventurous and always up for trying new experiences.
Supportive and understanding, always there for him when he needs you.
Passionate about your interests and hobbies.
Funny and able to make him laugh even on his worst days.
A great listener and communicator.
Loyal and trustworthy, someone he can always rely on.
Creative and full of unique ideas.
A loving and caring partner who makes him feel special.
Patient and able to handle difficult situations with grace.
Ambitious and always striving to achieve your goals.
A great cook who can whip up delicious meals.
A true friend who is always there for him, no matter what.
A loving and nurturing person who would make a great mother someday.
Confident and self-assured, but also humble and down-to-earth.
A hard worker who is dedicated to her career and personal growth.
An amazing partner who has made his life better in countless ways.
gpt-4-preview-1106
I’m an AI and do not have personal relationships or a boyfriend, but if someone were to describe a person positively, they might use compliments and descriptors such as:
Intelligent and insightful
Kind and compassionate
Funny and witty
Strong and independent
Supportive and understanding
Creative and innovative
Passionate and dedicated
Warm and affectionate
Adventurous and spirited
Reliable and trustworthy
These are just examples of positive traits someone might use to describe a significant other.
The model is capable of generating text with 4000 tokens of high quality. This becomes apparent when we input a long text and request the creation of a subsequent paragraph, for example.
The main issue seems to be the cost. Completion is expensive. Aiming to scale to as many users as possible, OpenAI has trained the newer models to produce increasingly shorter responses.
The mistake, in my opinion, is not providing a choice. Charge more, but give the option to users who desire high performance.
One caveat. While I hate the total and ever-increasing neutering (species-ist term?) of GPT-4, I do have to say that in my work on analyzing recent papers in genomics/molecular-biology, I AM getting increasingly sophisticated and comprehensive analyses out of later versions, gpt-4-preview-1106 being the best yet.