To increase throughput was looking to implement batching as per documentation. Chat completion endpoint is only returning response for first prompt unless I change the prompt according to workaround mentioned in s similar thread from March 2023.
Implementing above workaround is not ideal? Wanted to check if other folks have run into similar issue with chat completion endpoint.
Thanks