Extract response for each prompt within a single batch

API
,
1

Hello,

I want to send two prompts in a single batch and get the response to both of them. I use .jsonl file to create and submit the batch. This is the content of the .jsonl file:

{
   "custom_id":"{'name_module':'{name_module}','name_section':'{name_section}'}",
   "method":"POST",
   "url":"/v1/chat/completions",
   "body":{
      "model":"gpt-4o",
      "response_format":{
         "type":"json_object"
      },
      "messages":[
         {
            "role":"system",
            "content":"Please give the output as a JSON. You are a helpful assistant"
         },
         {
            "role":"user",
            "content":"Please see the following instructions!"
         },
         {
            "role":"user",
            "content":"Simply say hello!"
         },
         {
            "role":"assistant",
            "content":"Response to content."
         },
         {
            "role":"user",
            "content":"Simply say goodbye!"
         },
         {
            "role":"assistant",
            "content":"Response to content again."
         }
      ]
   }
}

The output is simply a ‘goodbye’ with no response to the 1st prompt (no ‘Hello’) in the response. The response is stored as JSON like this

{
   "id":"batch_req_KkJWGMnBmbKu4V1kTeDDpi0y",
   "custom_id":"{'name_module':'module','name_section':'section'}",
   "response":{
      "status_code":200,
      "request_id":"c5a503dbb1a1a99db6ba60a171173854",
      "body":{
         "id":"chatcmpl-9j9yMzdjGYEBB1supu6nqfPgdzd9u",
         "object":"chat.completion",
         "created":1720550122,
         "model":"gpt-4o-2024-05-13",
         "choices":[
            {
               "index":0,
               "message":{
                  "role":"assistant",
                  "content":"{"message":"Goodbye!"}"
               },
               "logprobs":"None",
               "finish_reason":"stop"
            }
         ],
         "usage":{
            "prompt_tokens":63,
            "completion_tokens":10,
            "total_tokens":73
         },
         "system_fingerprint":"fp_4008e3b719"
      }
   },
   "error":"None"
}

Currently I have only one choices value ('index':0). I expected to have two, one for the first prompt and one for the second prompt.

I also tried removing the following after each user prompt, but didn’t work either.

         {
            "role":"assistant",
            "content":"Response to content."
         }

Any suggestions? Thanks.