Batching with ChatCompletion not possible like it was in Completion

[sps]

UPDATE: @yotam.martin here’s how you can do batching with ChatCompletion

Batching with ChatCompletion Endpoint

Intro Ever since OpenAI introduced the model gpt-3.5-turbo, aka ChatGPT, to the OpenAI API on the Chat Completions endpoint, there has been an effort to replicate “batching” from existing users of the completions endpoint migrating to ChatCompletions - owing to the economical pricing. In the scope of this tutorial, we refer to combining multiple completion requests irrespective of the contexts into a single API call as batching. Why use batching? Instead of explaining this, I’ll quote from O…