Api Billing for streaming if I close connection midway

If I am using the streaming feature and I close the connection midway, will I be charged only by the tokens that I have consumed through the stream till now or will it be charged by the entire number of tokens that will be generated in the completion(tokens consumed till now + tokens not generated till now)


Yes, sort of.

Terminating the connection will end the generation. That said, there might still be 2 or 3 tokens in transit that you will be charged for.

That said, I can’t find this in the documentation anymore :thinking:

PS: Be advised that you have to make sure the connection to openai is actually cancelled/closed. Sometimes they just hang around if you’re not careful.

1 Like