Codex API returning finish_reason=null

API
1

Sample py code:

openai.Completion.create(
    model='code-davinci-002',
    prompt=['<|start|>1<|end|><|start|>2<|end|><|start|>'],
    max_tokens=10,
    stop=['<|end|>'],
    temperature=0
)

Around 20-30% of the time, it returns:

"choices": [
    {
      "finish_reason": null,
      "index": 0,
      "logprobs": null,
      "text": "3"
    }
  ]

Otherwise, it returns expected result:

"choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null,
      "text": "3"
    }
  ]

In some cases, for longer prompts, it fails ~100% of the time.

If start/end tags are changed from <|this|> to <this>, the problem never happens.

Any ideas why this might be the case? It seems like it started occurring today, since some older prompts I have which I know were working correctly just a couple of days ago, do not work anymore.

1 Like
2

Hi @ivan.bestvina

What is your expected completion using such a prompt as above?

:slight_smile:

3

I wrote the expected result above, which I get ~70% of the time. As far as completion goes, it seems to always be correct, but the finish_reason is sometimes, randomly, null instead of 'stop'.

4

I see, sorry I missed it

Expected result:

3

Are these all the prompt params you are using?

   model='code-davinci-002',
    prompt=['<|start|>1<|end|><|start|>2<|end|><|start|>'],
    max_tokens=10,
    stop=['<|end|>'],
    temperature=0
5

No problem, thanks for responding! Yes, I only used these ones, all the others are left default. I tried other params, but the problem seems to occur with frequency and similar parameters changed as well.

6

FWIW (not much).

If I remove the array from your prompt and stop, so far, I get 100% expected completions. Tried 10 times.

Example Output

Screenshot 2023-02-28 at 5.44.47 PM
Screenshot 2023-02-28 at 5.44.47 PM1820Ă—736 37.2 KB

Prompt Setup

Screenshot 2023-02-28 at 5.45.08 PM
Screenshot 2023-02-28 at 5.45.08 PM1862Ă—786 42.4 KB

Update: Tried 20 times, 100% expected results.

Probably does not help much, but just thought you might like to know.

:slight_smile:

7

Thanks for noting that! Sadly, I’m running a bunch of prompts in parallel, so I cannot remove the array. I just put a single example in here for simplicity. But maybe your info helps someone from OAI to fix the problem :slight_smile:

8

Yeah, gotcha… I noticed problems with arrays a few weeks ago.

No idea why either.

:slight_smile:

9

Just started having this same problem today, it’s happening on “length” as well.