openai.Completion.create() only ever returns ""

General API discussion
1

With a variety of adjustments suggested by several chatbots, the function only ever returns “” (the zero-length string)

I created a Jupyter notebook here: sam-pytube/openai.ipynb at master · InTEGr8or/sam-pytube · GitHub

The python function looks like this:

def get_summary(text: str, max_tokens, temperature=0.9, engine="davinci") -> str:
    max_total_tokens = 2049
    text = f"Please summarize the following text in {max_tokens} tokens: [{text}]"
    chunks = [text[i:i+max_total_tokens] for i in range(0, len(text), max_total_tokens)]
    max_summary_tokens = int(max_tokens / len(chunks))
    summaries = []
    for chunck in chunks:
        response = openai.Completion.create(
            engine=engine,
            prompt=text,
            max_tokens=max_summary_tokens,
            temperature=temperature,
            top_p=1,
            frequency_penalty=0,
            presence_penalty=0.6,
            stop=["\n", " Human:", " AI:"]
        )
        summaries.append(response.choices[0].text)
    summary = " ".join(summaries)
    return summary

How can I get it to return something more interesting?

2

Why do you have a loop when you are not using the variable “chunck” anywhere? You appear to be passing text instead of chunck.

  • try putting a \n\n after the “tokens:” part of your prompt (before the [{text}] )

  • if the loop does run a second time, you are cutting of the actual instruction where you ask it to summarize the following text. The second iteration just has random text that it will try an continue instead of summarizing.

  • the way you are cutting the text means the last words may be cut in half in the middle of the word. The first word in the next pass could only have half the letters too.

  • the stop setting wont do anything because you haven’t include the AI and Human text in your prompt. It doesn’t know you want a chatbot, and it also doesn’t know what labels to use for each party unless you tell it in the prompt (I can see you are not using a trained model - so it cant be figuring it out from that either)

  • you are “returning” summary but not printing it anywhere. Have you used a debugger or printed values while it is running through the loops so you can see what it is getting at various stages through the code.

1 Like
3

Those are all good points. I was trying to break a large message up but pasted some reverted code.

Printing the intermediate full response is a good idea, but the problem still looks the same. I took out the two stops but left the ‘\n’ one.

Still getting response.choices[0].text = ""

My code looks like this now:

def get_summary(text: str, max_tokens, temperature=0.9, engine="davinci") -> str:
    max_total_tokens = 2049
    text = f"Please summarize the following text in {max_tokens} tokens: \n\n[{text}]"
    chunks = [text[i:i+max_total_tokens] for i in range(0, len(text), max_total_tokens)]
    max_summary_tokens = int(max_tokens / len(chunks))
    print(f"max_summary_tokens: {max_summary_tokens}")
    summaries = []
    for chunk in chunks:
        response = openai.Completion.create(
            engine=engine,
            prompt=chunk,
            max_tokens=max_summary_tokens,
            temperature=temperature,
            top_p=1,
            frequency_penalty=0,
            presence_penalty=0.6,
            stop=["\n"]
        )
        print("response: ", response)
        summaries.append(response.choices[0].text)
    summary = " ".join(summaries)
    return summary

Response

max_summary_tokens: 100
response:  {
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null,
      "text": ""
    }
  ],
  "created": 1672363465,
  "id": "cmpl-6SyQDL0qc8hYiUWmVCyQjlEaLr11d",
  "model": "text-davinci-003",
  "object": "text_completion",
  "usage": {
    "prompt_tokens": 56,
    "total_tokens": 56
  }
}
4

Try the following:

  • remove the stop setting altogether. The AI may be trying to start the completion on a newline and so you might be stopping it before it starts

  • remove the presence_penalty and frequency_penalty for now (I dont think this is related)

  • print the value for chunk so you can a) check it has a value; and b) feed it into the Playground if the changes haven’t helped

  • finally, you could try adding \n\n to the end of each chunk before you call the api

2 Likes
5

One of those was the fix! (not sure which, yet.)

Thanks for the help. Now I can move on to bigger and better things and longer texts.

max_summary_tokens: 100
chunk: Please summarize the following text in 100 tokens: 

[This is a test of the emergency broadcast system. This is only a test. If this had been an actual emergency, you would have been instructed to do something. This concludes this test of the emergency broadcast system.]
response:  {
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null,
      "text": "\n\nThis is a test of the emergency broadcast system. No action is required; this is merely a practice run. It is designed to ensure readiness in the event of an actual emergency. This test has now concluded."
    }
  ],
  "created": 1672367261,
  "id": "cmpl-6SzPRLyUbX5tEHIId4m9a86DVQms9",
  "model": "text-davinci-003",
  "object": "text_completion",
  "usage": {
    "completion_tokens": 44,
    "prompt_tokens": 56,
    "total_tokens": 100
  }
}

(I kind of like the summary better than the original, for practice drills)

6

I think it was the stop setting and gpt sending a newline at the start of the completion

I’m glad it’s working now