Tokenizer and playground calculated a mismatch between the number of tokens and the bill for text-Davinc-003

[Foxalabs]

this may be of potential interest?

How to calculate the tokens when using function call

Thanks a lot. I’ve try and compare with the return from API. Then I got below. public static int tokens(String modelName, Object functionCall, List<Function> functions) { Encoding encoding = getEncoding(modelName); int sum = 0; if (Preconditions.isNotBlank(functionCall)) { if (functionCall instanceof JSONObject) { sum += tokens(encoding, functionCall.toString()); } } for (Function function : functions) { …

Might not be it exactly, but it’s in the same ballpark.