Why differing response structure on keywords?

API
1

My request looks like:

    openai.api_key = cfg['OPENAI']
    resp = openai.Completion.create(
        model="text-davinci-003",
        prompt=f"Extract keywords from this text:\n\n{digest}",
        temperature=0.5,
        max_tokens=60,
        top_p=1.0,
        frequency_penalty=0.8,
        presence_penalty=0.0
    )

With one digest, I get something like:

  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "logprobs": null,
      "text": "\n\n-kword1 \n-kword2 \n-kword3 \n-kword4 \n-kword5 \n-kword6"
    }
  ],

and with another digest, I get something like:

  {
  "choices": [
    {
      "finish_reason": "length",
      "index": 0,
      "logprobs": null,
      "text": "\n\nKeywords: kword1, kword2, kword3, kword4, kword5, kword6, kword7"
    }

Why is there a different structure for the text returned? A different delimiter and , in one case, a “Keywords:” header but not in both cases? Why?

2

It’s response will always be slightly different, especially when it is given freedom in its answer. The differences in stop is only to do with your token parameter.

What you can do is provide some examples for a more consistent response. I prefer it much more than explicit instructions

3

“What you can do is provide some examples for a more consistent response.”

Hm. I am not sure what you mean here. I am trying to see if OAI is able to give keywords for proposed legislation. So I have digest text as the prompt, It is giving somewhat reasonable results.

I am not sure why, though, the difference in ‘finish_reason’ value would bring about a change in the structure of the response. It seems odd.

For all of these queries, I am starting the prompt with: “Extract keywords from this text:”

Might it work better to say : “Extract a list of tab-separated keywords from this text:”. Or is there some other magic words to say that would lead to a consistent structure of the choices-text response string?

FYI, asking for the tab-separating list gives me:

“The model predicted a completion that begins with a stop sequence, resulting in no output. Consider adjusting your prompt or stop sequences.”

This reminds me of the old joke: How many prolog programmers does it take to screw in a light bulb? the answer: False.

4

What I was referring to is “few-shot learning”.

This simply means that the model has output nothing because your stop sequence was the first thing that it “wrote”. Are you by any chance using “\n” as a stop sequence?