I have trained my custom chat model with PDF, and I am using OpenAIEmbeddings, and I am not able to get total_tokens, prompt tokens while I am generate embeddings, on the other hand if use openai embeddings api, tokens are present there, how to get tokens by using OpenAIEmbeddings from Langchain
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Overheads 'tokens'? Usage does not add up | 3 | 1341 | September 20, 2023 | |
| How to calculate the cost and tokens when using Faiss vector database with openai through LangChain | 9 | 7307 | December 18, 2023 | |
| Embeddings for tokens used by GPT models? | 2 | 934 | December 17, 2023 | |
| Exception in embedding creations | 0 | 1250 | November 15, 2023 | |
| Langchain app which reads PDFs using openai embeddings and model | 2 | 2548 | December 13, 2023 |