Understanding billing of usage

It was a real eye-opener for me to discover how the billing process works! I went on a little adventure, conducting an experiment that cost just $0.22. But here’s the kicker - if I send or receive 1001 tokens, there’s an extra charge of $0.11 for that one sneaky last token. Can you imagine? When dealing lots of data, that can tally up to a pretty penny, and all for something that doesn’t seem to add too much to the mix. I’m super eager to hear how you folks navigate this little twist!

Semantic cleanup of a text with about same content size both way, was charged $0.22.

“choices”: [
“finish_reason”: “stop”,
“index”: 0,
“message”: {
“content”: “Part Four \u2013 Explanatory Notes\n\nSection A: Agriculture, Hunting and Forestry\n\n01 Agriculture, Hunting and Related Service Activities\n\n01.1 Growing of Crops; Market Gardening; Horticulture\n\n01.11 Growing of Cereals and Other Crops N.E.C.\n\nThis class includes:\n\n- Growing of cereal grains: hard and soft wheat, rye, barley, oats, maize, rice, etc.\n- Growing of potatoes\n- Growing of sugar beet\n- Growing of tobacco, including its preliminary processing: harvesting and drying of tobacco leaves\n- Growing of oilseeds or oleaginous fruit: peanuts, soya, colza, etc.\n- Production of sugar beet seeds and forage plant seeds (including grasses)\n- Growing of hop cones, roots and tubers with a high starch or inulin content\n- Growing of cotton; growing of diverse textile plants; retting of plants bearing vegetable fibres\n- Growing of dried leguminous vegetables such as field peas and beans\n- Growing of plants used chiefly in pharmacy or for insecticidal, fungicidal or similar purposes\n- Growing of crops n.e.c.\n\nThis class excludes:\n\n- Growing of melons, see 01.12\n- Growing of sweet corn, see 01.12\n- Growing of other vegetables, see 01.12\n- Growing of flowers, see 01.12\n- Production of flower and vegetable seeds, see 01.12\n- Growing of horticultural specialities, see 01.12\n- Growing of olives, see 01.13\n- Growing of beverage crops, see 01.13\n- Growing of spice crops, see 01.13\n- Growing of nuts, see 01.13\n- Gathering of forest products and other wild growing material, see 02.01\n- Growing of vegetable materials used for plaiting, see 02.01\n\n01.12 Growing of Vegetables, Horticultural Specialities and Nursery Products\n\nThis class includes:\n\n- Growing of vegetables: tomatoes, melons, onions, cabbages, lettuce, cucumbers, carrots, beans, cress, sweet corn, courgettes, egg-plants, leeks\n- Growing of seasoning herbs and vegetables: capers, \u2019peppers\u2019, fennel, parsley, chervil, tarragon, sweet marjoram\n- Growing of mushrooms, gathering of forest mushrooms or truffles\n- Growing of flowers\n- Production of seeds for flowers, fruit or vegetables\n- Growing of plants for planting or ornamental purposes, including turf for transplanting\n\nThis class excludes:\n\n- Growing of potatoes, see 01.11\n- Growing of sugar beets, see 01.11\n- Growing of oilseeds and oleaginous fruit, see 01.11\n- Growing of roots and tubers with a high starch or inulin content, see 01.11\n- Growing of cotton or other vegetable textile materials, see 01.11\n- Growing of spice crops, see 01.13\n- Growing of olives, see 01.13\n- Operation of forest tree nurseries, see 02.01\n- Growing of Christmas trees, see 02.01\n\n01.13 Growing of Fruit, Nuts, Beverage and Spice Crops\n\nThis class includes:\n\n- Production of fruit: apples, pears, citrus fruit, apricots, strawberries, berries, cherries, peaches, bananas, avocados, dates,”,
“role”: “assistant”
“created”: 1689010997,
“id”: “chatcmpl-7apCncfGToujfF6EOc1jqZED7hWB1”,
“model”: “gpt-4-0613”,
“object”: “chat.completion”,
“usage”: {
“completion_tokens”: 736,
“prompt_tokens”: 756,
“total_tokens”: 1492

Well, while true, that does not tell the whole story, sure, that 1 token over 1k is charged for, but the next 999 are not, you pay per 1k used for the month. It’s not like it resets every time, you pay at the end of the month for the full token count so, sure if you have used 1001 for the entire month… you will pay for for 2k.

Hrm. Being under a grant, it seems there’s no way to get a per-request figure “billed”. However, the apexcharts column bar chart in “usage” has daily figures for “val” that the mouseenter (mouse over tooltip) function rounds to the penny.

Going back to a day with only one API request:
gpt-3.5-turbo-0613, 1 request
13 prompt + 171 completion = 184 tokens

“inspect element” on the chart:

That’s $0.0019646 per 1k total.

In line with price list $0.0015 / 1K input tokens $0.002 / 1K output tokens:
0.0000195 in + 0.000342 out = 0.0003615

(Also some “model-detector-2023-01-25-v2” with only one other unanswered question about that on the internet. Probably getting billed or at least noted for calling deprecated models…)

Now the math needed:

 def give_forum_users_api_credit(solution, answer):
     credit = 0.0
     if solution:
         credit = 0.1
         for token in answer:
             perplexity_score = perplexity_weight(token)
             credit += perplexity_score
     return credit
1 Like

So here’s the scoop: I did this test because I was scratching my head over why the cost wasn’t lining up with what I had anticipated. I did a semantic clean-up job on a text file of about 100 pages and my bill came in at more than double the total amount of tokens I had processed. Now, isn’t that a head-scratcher?

Turns out, it looks like it’s calculated per request. So, if you make 100 requests of 1001 tokens, you’ll be billed for 100x2000x$0.11, not the 100x1001x$0.11 that you might expect. Quite the puzzle, isn’t it? I’m all ears if you’ve got thoughts or experiences to share on this one!

Oh, this has definitely piqued my interest! Next time I’m tackling a big cleanup, I’m totally going to put your ideas to the test. Let’s see how it all pans out. I’m really looking forward to it!

Hi @nikolaimanigoff

The pricing on gpt-4 is different for prompt and completion tokens:

Model Input Output
8K context $0.03 / 1K tokens $0.06 / 1K tokens
32K context $0.06 / 1K tokens $0.12 / 1K tokens

Based on the above usage data shared by you, that request would have been billed as follows:
0.736 x 0.06 + 0.756 x 0.03

Well, isn’t this a lightbulb moment! Thanks a bunch for clearing that up. I guess I should have given the documentation a more thorough read. My bad entirely. This one’s definitely on me. Thanks again!