I have developed a tool utilizing OpenAI’s GPT-3.5-turbo-1106, integrated with LangSmith and LangGraph. I am looking to implement a feature that limits the usage of my bot based on a certain cost threshold for OpenAI API calls. Specifically, I want to allow users to make free requests until they reach a $5 monthly limit, similar to how GPT limits requests for free users to access their gpt4o model
Im using langGraph
#For final response
from langchain_core.messages import HumanMessage
inputs = {
“messages”: [
HumanMessage(
content=‘Can you delete the user with email mike@gmail.com.’
)
]
}
Use the Runnable to get the final state
final_state = app.invoke(
inputs,
config={“configurable”: {“thread_id”: 42}}
)
Print the final response
print(final_state)
The final_state doesn’t provide any information regarding tokens.
I tried using get_openai_callback function but it didnt work either.
How do i get to know the tokens usage per api call so that I can restrict the user after they reach their limit.