If I am using the streaming feature and I close the connection midway, will I be charged only by the tokens that I have consumed through the stream till now or will it be charged by the entire number of tokens that will be generated in the completion(tokens consumed till now + tokens not generated till now)
Welcome!
Yes, sort of.
Terminating the connection will end the generation. That said, there might still be 2 or 3 tokens in transit that you will be charged for.
That said, I can’t find this in the documentation anymore 
PS: Be advised that you have to make sure the connection to openai is actually cancelled/closed. Sometimes they just hang around if you’re not careful.
1 Like