I want an OpenAI model (via API) to faithfully continue a generation that I have in mind.
For example:
User: “What’s your favorite food?”
Assistant Prefix (provided by me): “Cake! The reason why is because”
and then have the model continue generating " I like how it tastes!" (or whatever it might choose to complete the generation with).
Analogously, if I were “raw prompting,” I’d like to do something like
p = “”“<BOS_TOKEN><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>What’s your favorite food?<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>It’s Cake! The reason why is because”“”
Vaguely similarly, Claude has an ability to “prefill” the generation, which seems perhaps like what I’m looking for <Google “Prefill Claude Response”>
Does a similar feature exist for the OpenAI API (either a raw prompting ability or a generation prefill feature)?
I’m not looking for some hacky prompting solution. This is for a benchmark that I’m trying to work on 
. Thanks!
2 Likes
As of now (and hopefully only temporarily) this isn’t supported with newer models that use ChatML (Any model in the ChatCompletions category)
The latest one available would be 3.5-turbo-instruct.
You can try it out on the playground
So, yes, but the format that support it (Completions) is considered legacy
4 Likes
Ah, shame – Is that feature available just for the legacy models in playground, or can I do it via API? Just a convenience question for me – thanks for the information.
Also curious if that example you showed is really parsing the “roles” correctly?
Anything in the playground can be done in the API.
You just would need to use the Completions endpoint
https://platform.openai.com/docs/api-reference/completions/create
Not really. Just autocompleting. I set a stop sequence (“\n”) to stop it from adding more lines.
1 Like
Just append the instruction to the user message:
User message:
Hi how are you doing .
Assistant Response should always start and continue response message from : {“assistant_response”:”
Assistant response will be
{“assistant_response”:”I’m doing well, Daniel! How about you? How can I assist you today?”}
