Short description
If you ask “gpt-4” or “gpt-4-1106-preview” models to recite the famous “Blue Dot” quote by Carl Sagan, it fails to do so. Instead, the model almost always cuts it on one particular place:
“everyone you love, everyone you know, everyone you’ve”
This happens almost always. Looks like some kind of a repetition detector is being too stringent.
Steps to reproduce
Run the code below.
The expected result: the code prints the full quote, ending it with the word “civilization”.
The observed result: it ends in a middle of the quote, with “everyone you’ve”.
import os
from openai import OpenAI
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
def format_text(raw_text):
text_for_gpt = f"Please format this text: {raw_text}"
completion = client.chat.completions.create(
model="gpt-4-1106-preview",
messages=[
{"role": "system", "content": "You are a helpful assistent."},
{"role": "user", "content": text_for_gpt},
],
)
return str(completion.choices[0].message.content)
blue_dot_text = """
Carl Sagan said: Look again at that dot.
That's here, that's home, that's us.
On it, everyone you love, everyone you know,
everyone you've ever heard of,
every human being who ever was
lived out their lives.
the aggregate of our joy and suffering,
thousands of confident religions,
ideologies and economic doctrines,
every hunter and forager, every hero
and coward, every creator and destroyer of civilization
"""
result = format_text(blue_dot_text)
print(result)
Sample erroneous output:
BTW, it’s not only the API. The GPT4 in the default web interface stops on roughly the same spot, and complains about policy violations.
