I’m trying to extract info from a bunch of input texts with langchain and gpt3.5. Instead of using the same prompt template for the instructions and substituting the input text each time, I’m thinking of storing the first input and output into a ChatTokenBufferMemory, so that in later calls of openai api I can just say “can you do the same to this text”. But do the tokens in ChatBufferMemory also cost money? My original intention is to save money because “can you do the same to this text” has way less tokens than my whole prompt template with the instructions, but I’m wondering that the chat history is also fed into api call each time?
I think you’d be better off asking this in a LangChain forum.
But, in general you get charged for all tokens, every time, so…
Yes, you pay for those tokens too.