Hello OpenAI Community,
We’re developing a desktop application targeting C-suite executives that uses the GPT-4o API for ongoing contextual assistance. Our application needs to maintain conversation context across multiple sessions while being mindful of token usage and response quality.
Specifically, we need advice on:
- What’s the optimal approach for managing conversation history with GPT-4o when executives may have intermittent conversations spanning days or weeks?
- Are there recommended token management strategies to balance context retention with cost efficiency? (Currently considering window-based truncation and summary techniques)
- What’s the best way to handle topic transitions within the same conversation thread without losing relevant context?
- Has anyone implemented effective context persistence techniques that don’t rely solely on passing the full conversation history each time?
- Are there specific prompt engineering approaches that work well with GPT-4o to help it retrieve relevant information from earlier in the conversation?
Any insights from those who’ve built similar executive-facing tools would be much appreciated.