Hi there, is there a way to get the token count of a completion as part of the API call or in a separate call? Like a programmatic version of our usage page so we can tie usage amount to users?
3 Likes
It would be great if there was a pre-tokenizer endpoint or something like that.
I did what you ask adding echo:true in api call via curl and in response I get back this where I should look for token count logprobs is null here
OK now I get you in this case I also need to provide logprobs=1 don’t know why??
curl https://api.openai.com/v1/engines/davinci/completions
-H “Content-Type: application/json”
-H “Authorization: Bearer YOUR_TOKEN”
-d ‘{
“prompt”: “The following is a conversation with an AI assistant. The assistant is helpful, creative, clever, and very friendly.\n\nHuman: Hello, who are you?\nAI: I am an AI created by OpenAI. How can I help you today?\nHuman: “,
“temperature”: 0.9,
“max_tokens”: 150,
“top_p”: 1,
“frequency_penalty”: 0,
“presence_penalty”: 0.6,
“stop”: [”\n”, " Human:", " AI:"],
“echo”:“true”,
"logprobs:1
}’
but the problem is I don’t want the prompt+completion because of the obvious reasons let say I designed some prompts so I don’t want to count those as used tokens for my users
1 Like
hey that’s just an one case but there are many but I find in docs
may this is can be more simpler to use I
@jloganolson I asked that in feature requests a while ago, no update yet, but hoping Feature request: Total tokens used in request - #3 by sergeliatko
2 Likes