Strategies to reduce conversational GPT-4 costs

Hey there, we are building an interactive bot that can interact for a conversation of about 10 messages.

In our current design we use the GPT4 APIs and for every step of the conversation we send the prompt + all the messages up until that point. This is pretty terrible since: (1) I end up paying the cost of the prompt for every user, (2) as conversations continue the price per message increases a lot!

  • Are there ways to go around problem (1)? Is it possible to amortize the cost of one prompt across multiple conversations?
  • Are there ways to avoid problem (2)? Are there ways to avoiding sending the entire conversation up until that point?

You should think about using tools like Langchain and. pinecone,