The Completions API doesn't *really* return completions

API
1

If I send this prompt for completion:

"The following is the list of the descending positive integers starting from 9. The list is complete. 9 8 7 6 5 4 3 2 "

I believe the completion of this ought to be “1”, and nothing more. However, AsyncOpenAI.chat.completions.create is more interested in having a chat:

“The list you’ve provided is a sequence of descending positive integers starting from 9 and decrements by 1 each time. If the list is complete with respect to only including single-digit positive integers 9 down to 2, then the sequence you’ve given is accurate. However, if the sequence is meant to include all positive integers down to 1 in a descending order, then the number 1 is missing at the end of the list. It should be:\n\n9 8 7 6 5 4 3 2 1\n\nEach number in the sequence is one less than the number before it, and this sequence is typically known as a countdown. The integer 1 is the smallest positive integer and would complete the sequence of single-digit positive integers from 9 down to 1.”

Now to the LLM’s credit, it knows what the list should contain. but I would not describe its response as a completion of my input. Though my knowledge of LLMs is limited, I imagine the most likely predicted next token following my input is “1”, but whatever layers reside between my API call and the transformer are complicating things in undesirable ways for this use case. Is there a way to achieve a sort of “raw” mode? Thanks.

2

Welcome to the community!

Are you using gpt-3.5-turbo-instruct?

Are you limiting max_tokens? Might also ask for completion in the system prompt.

2 Likes
3

Hi Paul. Here’s my invocation.

task = _remote_client().chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": content}],
    temperature=1,
    max_tokens=4096,
    top_p=1,
    frequency_penalty=0,
    presence_penalty=0,
)

Might also ask for completion in the system prompt.

Sorry, would you elaborate? I don’t know what that is.

4

GPT-4 is unfortunately just a chat model

instruct are the actual completion models:

The image shows a text snippet from a black-background interface labeled "Playground," stating a list of descending positive integers from 9 to 1. (Captioned by AI)
The image shows a text snippet from a black-background interface labeled "Playground," stating a list of descending positive integers from 9 to 1. (Captioned by AI)1177×265 9.19 KB

you can play with it here:

https://platform.openai.com/playground/complete

Is that what you meant?

4 Likes
5

You need to give it a few shots if you want an exact response like that, at least one or two. Look up few shot prompting.

6

I see. My example in the my original post is of course just a contrived toy example. What I’m really after is completion of Python code. I think I’ll be better off trying to programmatically cope with GPT-4’s difficult behavior in exchange for its superior ability.

7

In my real use case (not the toy example I gave), I gave it 50 examples. xD

8

Not sure about 50 examples, I usually have the responses reduced down to a single number sequence with one shot. Also, I usually add in a line that says you will fail if you respond with anything except the single number.

1 Like
9

Ah, yes, that’s a Chat Completion model, not Completion (Legacy)…

/v1/chat/completions All GPT-4o, GPT-4o-mini, GPT-4, and GPT-3.5 Turbo models and their dated releases. chatgpt-4o-latest dynamic model. Fine-tuned versions of gpt-4o, gpt-4o-mini, gpt-4, and gpt-3.5-turbo.
/v1/completions (Legacy) gpt-3.5-turbo-instruct, babbage-002, davinci-002

More info here…

https://platform.openai.com/docs/models/model-endpoint-compatibility

1 Like
10

Did you try adding “no prose” in your prompts.

Also try using user role instead of system.

Also maybe add a label such as

Output:

Right at the end of the prompt to indicate the output can force it to do the minimal without explanation (prose).

11

Try something like this:

predict the next number in this sequence. put the next number in a <next_number> tag

9, 8, 7, 6, 5, 4, 3, 2

1 Like