Why can't I access my input to my model in the chat completion API?

API
, ,
1

When I pull all my chat completions from here: https://api.openai.com/v1/chat/completions

the standard response only contains the choices, nothing related to the input that was given.

For context I am trying to produce a jsonl file using my chat completions for fine-tuning of my models.

Any help on this would be appreciated, also maybe I’m missing something but a chat completion api without the input feels kind of incomplete

2 Likes
3

You need to retrieve the chat messages with an additional step.

The python example in there has a little mistake, the array of messages is in completions.data. Here is a fixed example:

from openai import OpenAI
client = OpenAI()

completions = client.chat.completions.list()
first_id = completions.data[0].id
first_completion = client.chat.completions.retrieve(completion_id=first_id)
messages = client.chat.completions.messages.list(completion_id=first_id)
print(messages)
4 Likes
4

Having an API that sends thousands of words that you sent right back at you would seem silly, don’t you think?

Here’s producing a JSONL file using chat completions. I was going to show how simple it was, but then I told an AI to make in robust and harder to break and fixed what it broke.

A function for making a training example line

from openai import OpenAI
client = OpenAI(timeout=90)

import json

def make_jsonl_entry(input_messages, api_response, tools):
    # Get the assistant message object
    msg = api_response.choices[0].message
    # Use model_dump() if available, else fallback to __dict__
    msg_dict = msg.model_dump() if hasattr(msg, "model_dump") else msg.__dict__
    # Only keep the keys you want, and handle tool_calls/content being optional
    entry = {
        "role": msg_dict["role"],
    }
    if "content" in msg_dict and msg_dict["content"] is not None:
        entry["content"] = msg_dict["content"]
    if "tool_calls" in msg_dict and msg_dict["tool_calls"]:
        entry["tool_calls"] = msg_dict["tool_calls"]
    # If you always want tool_calls present (even if empty), uncomment:
    # entry["tool_calls"] = msg_dict.get("tool_calls", [])

    return json.dumps({
        "messages": input_messages + [entry],
        "parallel_tool_calls": False,
        "tools": tools,
    })

Carry on with your normal usage:

input = [
    {"role": "system", "content": "You are a helpful language assistant."},
    {"role": "user", "content": "Say 'banana' 10 times!"},
]
tools = []
api_response = client.chat.completions.create(
  model="gpt-4-turbo-2024-04-09", max_tokens=5, messages=input, tools=tools,
)
print(f"AI said:\n{api_response.choices[0].message.content}")

jsonl_chat_to_save = make_jsonl_entry(input, api_response, tools)

print(f"\n\nYour training file line:\n{jsonl_chat_to_save}")

If you simply must get what you just sent from an API, you’ve got the Responses endpoint. The data return is not your friend, though (nor is an API that makes you pay for more than the desired output length.)


r_input = [
    {
        "role": "system",
        "content": [
            {"type": "input_text", "text": "You are a helpful language assistant."}
        ]
    },
    {
        "role": "user",
        "content": [
            {"type": "input_text", "text": "Say 'banana' 10 times!"}
        ]
    }
]
response = client.responses.create(
  model="gpt-4-turbo-2024-04-09",
  input=r_input,
  max_output_tokens=16,
)
print(json.dumps(response.model_dump(),indent=2))  # can't trust index 0
input_get = client.responses.input_items.list(response.id, order="asc")
print(f"retrieved inputs: {input_get.model_dump()}")  # not chatC items
5

@_j maybe I didn’t explain my use case better, I am using a multi agent system from the oAI libraries so I am not making these create api calls myself but rather just storing the chat completions. In that case having the input also would be super beneficial, thank you for your help.

@aprendendo.next that API call works perfectly! thanks so much!

3 Likes
6

@aprendendo.next I’m facing another issue where when I get the messages from that api I can’t see the tool/function calls made in some cases

im getting this in the messages body:

{
			"id": "id123",
			"role": "tool",
			"content": "None",
			"content_parts": null,
			"name": null
		}

content, name everything is null, it just says tool

7

@sitanshu I’ve just tried here with a function call and it seems to be a limitation of the API.

Unfortunately that’s all it returns, as of now I don’t know if there is another way to retrieve the exact inputs.

This works for text, but as you’ve noticed it has some limitations for other input types.

8

made a post about it here: Trying to access my chat completion input in a multi agent architecture