Failed to calculate number of tokens, falling back to approximate count

API
, , ,
1

Hi guys, I wanted to ask you does anyone know where is the problem with this because Im getting an error: (Failed to calculate number of tokens, falling back to approximate count) after trying to make a request to open ai by using langchain and sql database connection. On the request I can see that there is a property with max_tokens: 256, is there a problem with the api key or pricing plan or anything related to the database size?

2

Hi, John

This seems to be solely a Langchain issue, have you tried asking on their support forums?

3

Hi @Foxabilo, I didn’t found any solution for it on langchain forum this is the reason that Im asking.

4

I think it has something to do with network, you can try to use WARP

5

Hi @miyutest1,
What do you mean with use WARP?

6

It‘s a vpn, you can look at 1.1.1.1, and you can replace it with other
Count token need network request，you can try the following command:

curl API/v1/models -H “Authorization: Bearer API_KEY”

Check if the request was successful

7

You can try to use js-tiktoken directly, which is what langchain currently uses.