Hi,
I need a bit of help with my custom GPT. I want my GPT to stream responses from my server. The response is a task execution log. It is not too big but takes time, probably more than an hour. The problem is, the GPT reports an error after a few seconds, let’s say 20 or so. I’m quite sure I provide the chunks. Any insights?
What is your experience streaming chat from external servers to the GPTs?
I’ve confirmed the GPT start to receive the firs few chunks. Here is my response type:
http_response response(status_codes::OK);
response.headers().add(U(“Content-Type”), U(“application/json”));
response.headers().add(U(“Transfer-Encoding”), U(“chunked”));
response.set_body(concurrency::streams::istream(*streambuf));
Many thanks!