Creating a Continuity-Based Chatbot similar to chatGPT

API
, ,
1

Hi everyone,

I’m working on developing a chatbot similar to OpenAI’s ChatGPT. However, I’ve encountered a couple of challenges due to the nature of the GPT-4 model, which doesn’t retain the memory of previous requests. To create a continuous chat experience, I have to send both the user’s and AI’s previous conversions. This approach presents two main issues:

  1. Token Limit: As conversations progress, the message body grows larger, which can lead to exceeding the maximum token limit.

  2. Cost Implications: Increasing the number of tokens means higher costs for us, as we incur more expenses based on token usage.

I’d appreciate any insights or suggestions on how to address these issues.

Thanks!