I’ve been looking into how we could further enrich the data returned from a request to the completions endpoint. One piece of information that might be highly beneficial is the inclusion of the maximum available tokens in the meta information of the response.
Currently, the endpoint’s response already includes useful meta information such as the finish_reason. However, having extra data about the available tokens could aid us in effectively making follow-up requests or better managing our token usage.
There’s already a palpable demand for this feature from the user community, as evidenced by discussions such as this post on the community page. Unfortunately, there isn’t a practical workaround available to them at the moment.
Given the circumstances, I believe incorporating this feature directly into the endpoint’s response could greatly improve its usefulness. Looking forward to your thoughts on this.