How to see the final formatted prompt sent to an LLM?

API
1

How can I see the final formatted prompt sent to the LLM using the chat.completions python client? I dont see this in the documentation anywhere

2

You’re creating the prompt, so you should know what it is?

Do you have an example of what you mean?

If you haven’t yet, check out the OpenAI Quickstart guide that covers a lot of the basics…

3

@PaulBellow no I mean each model using a different prompt format <|im_start|> or [INST]/ etc.etc. the client abstracts this away with the roles in the messges

4

You do not see the actual tokens placed into context by the messages containers and the prompt for where the assistant shall write its response.

That internal communication is only between the chatCompletions API backend and the model.

OpenAI further doesn’t let you see what the AI is writing to call functions, and you can only place assistant messages in history by a similar function abstraction.

The AI can repeat back a lot of context by instruction solely focused on jailbreaking it, but it cannot reproduce special tokens and is reluctant to precisely replay other formatted context.

This tokenizer has a template for GPT-4 that is mostly correct for all models except original gpt-3.5-turbo-0301.

1 Like