# Token usage when using openai.chat.completions.create stream: true

**URL:** <https://community.openai.com/t/token-usage-when-using-openai-chat-completions-create-stream-true/469928>\
**Category:** API\
**Tags:** gpt-4, token\
**Created:** [November 4, 2023, 10:06am UTC](https://community.openai.com/t/token-usage-when-using-openai-chat-completions-create-stream-true/469928 "2023-11-04T10:06:28Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![SoftTimur](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/softtimur/32/7727_2.png) [@SoftTimur](https://community.openai.com/u/SoftTimur)\
**Post date:** [November 4, 2023, 10:06am UTC](https://community.openai.com/t/token-usage-when-using-openai-chat-completions-create-stream-true/469928/1 "2023-11-04T10:06:29Z")

</div>

Hello,

Similar questions may be asked previously, but I did not find an answer for our case.

We have an application using ReactJS and NodeJS. We send GPT-3.5 and GPT-4 requests to OpenAI by `openai.chat.completions.create` and `stream: true`. We would like to count the tokens of prompts and complements as correct as possible to give a precise billing.

Does anyone know how to get or calculate this token usage? Which libraries can be used?

Thank you

---

<div class="post-metadata">

**Author:** ![\_j](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/_j/32/766292_2.png) [@\_j](https://community.openai.com/u/_j)\
**Post date:** [November 4, 2023, 10:11am UTC](https://community.openai.com/t/token-usage-when-using-openai-chat-completions-create-stream-true/469928/2 "2023-11-04T10:11:52Z")

</div>

> **[@dqbd/tiktoken](https://www.npmjs.com/package/@dqbd/tiktoken)**
>
> Javascript bindings for tiktoken. Latest version: 1.0.7, last published: 6 months ago. Start using @dqbd/tiktoken in your project by running \`npm i @dqbd/tiktoken\`. There are 146 other projects in the npm registry using @dqbd/tiktoken.

> **[js-tiktoken](https://www.npmjs.com/package/js-tiktoken)**
>
> JavaScript port of tiktoken. Latest version: 1.0.7, last published: 5 months ago. Start using js-tiktoken in your project by running \`npm i js-tiktoken\`. There are 56 other projects in the npm registry using js-tiktoken.

Ensure you use the `cl100k-base` token encoder for chat models.

The OpenAI official/authoritative tiktoken is in Python.

---

<div class="post-metadata">

**Author:** ![SoftTimur](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/softtimur/32/7727_2.png) [@SoftTimur](https://community.openai.com/u/SoftTimur)\
**Post date:** [November 4, 2023, 10:14am UTC](https://community.openai.com/t/token-usage-when-using-openai-chat-completions-create-stream-true/469928/3 "2023-11-04T10:14:46Z")

</div>

Thank you.

It seems that js-tiktoken points also to [github.com/dqbd/tiktoken](https://github.com/dqbd/tiktoken), are @dqbd/tiktoken and js-tiktoken the same project?

---

<div class="post-metadata">

**Author:** ![\_j](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/_j/32/766292_2.png) [@\_j](https://community.openai.com/u/_j)\
**Post date:** [November 4, 2023, 10:28am UTC](https://community.openai.com/t/token-usage-when-using-openai-chat-completions-create-stream-true/469928/4 "2023-11-04T10:28:33Z")

</div>

Looks like one just has a few different commits, and both have dependencies. They are from the same developer, also here:

> <https://github.com/dqbd/tiktoken/blob/main/js/README.md>

---

<div class="post-metadata">

**Author:** ![SoftTimur](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/softtimur/32/7727_2.png) [@SoftTimur](https://community.openai.com/u/SoftTimur)\
**Post date:** [November 4, 2023, 10:50am UTC](https://community.openai.com/t/token-usage-when-using-openai-chat-completions-create-stream-true/469928/5 "2023-11-04T10:50:52Z")

</div>

So I should try first js-tiktoken, right?

---

<div class="post-metadata">

**Author:** ![\_j](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/_j/32/766292_2.png) [@\_j](https://community.openai.com/u/_j)\
**Post date:** [November 4, 2023, 10:52am UTC](https://community.openai.com/t/token-usage-when-using-openai-chat-completions-create-stream-true/469928/6 "2023-11-04T10:52:38Z")

</div>

Personally, if you have the server that can run it, I’d set up an API for tiktoken in python, and just make authenticated or internal calls to it.

---

<div class="post-metadata">

**Author:** ![SoftTimur](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/softtimur/32/7727_2.png) [@SoftTimur](https://community.openai.com/u/SoftTimur)\
**Post date:** [November 4, 2023, 11:02am UTC](https://community.openai.com/t/token-usage-when-using-openai-chat-completions-create-stream-true/469928/7 "2023-11-04T11:02:18Z")

</div>

Do you mean setting up a server which hosts openai/tiktoken in Python, and provides an API which can be requested by my backend?

---

<div class="post-metadata">

**Author:** ![\_j](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/_j/32/766292_2.png) [@\_j](https://community.openai.com/u/_j)\
**Post date:** [November 4, 2023, 11:16am UTC](https://community.openai.com/t/token-usage-when-using-openai-chat-completions-create-stream-true/469928/8 "2023-11-04T11:16:48Z")

</div>

Yes, that to go along with the server that sends client software requests to OpenAI, so you don’t do silly things like putting API keys in your app to be stolen and have your account emptied.
