I hope this message finds you well. I am reaching out for guidance regarding an issue I’ve encountered with the management of context tokens in assistant threads. I have observed that the “context tokens” count in my assistant threads is significantly high, leading to increased associated costs. Th…

[image] yako: I have observed that the “context tokens” count in my assistant threads is significantly high, leading to increased associated costs. This seems to be due to the accumulation of older, unnecessary messages in the threads. Is there a way to selectively delete old messages from thes…

[image] How to limit the number of messages or tokens that are persisted in a thread to maintain context in Open AI Assistants? API There is no “delete messages in a thread” API method. https://platform.openai.com/docs/api-reference/messages You get delete as…

They recently updated the documentation indicating that they are “exploring” the concept of not having a potential of >$1/message. Currently, the Assistant will include the maximum number of messages that fit in the context length. We plan to explore the ability for you to control the input / outp…

[image] logankilpatrick: The context on why this is the starting point is that ChatGPT uses a similar heuristic. We know this is not perfect and are exploring other options, many folks have asked for this so stay tuned! Thanks for the insight. Looking forward to the future of Assistants.

Hello @logankilpatrick , you might consider looking at the community-maintained Openai-php/client Git Lib: GitHub - openai-php/client: ⚡️ OpenAI PHP is a supercharged community-maintained PHP API client that allows you to interact with OpenAI API. . It has the feature to delete a message in a thread …

Good call. Looks like there’s an undocumented/beta API endpoint. # delete a message def delete_message_gpt(thread_id, message_id): # have to hit this one directly, as it's unpublished/beta url = f"https://api.openai.com/v1/threads/{thread_id}/messages/{message_id}" headers = { "…

Thanks, looking forward to this update since my context tokens are crazy higher. [image]

Hello, I noticed that you mentioned a beta/undocumented endpoint for deleting messages in a ChatGPT thread of assistant api. Have you had a chance to test this endpoint? If so, did it work as expected?

Hi … it does work, but gave me some unexpected results like responding to messages I thought were deleted. I didn’t spend much more time on making it work, as I didn’t want to have a production system depending on an undocumented endpoint. My suspicion is that if you delete a message in the middle …

It seems to work but leaves the function calls and their outputs including code_interpreter responses. However much you try to delete them with their IDs nothing will happen

Has anyone else tried this undocumented beta endpoint to delete messages because I want to give it a try so I can do anything to reduce my Contacts tokens

Reducing Context Tokens in Assistant Threads

API

fra_ab December 1, 2023, 3:56pm 2

Topic		Replies	Views
How to limit the number of messages or tokens that are persisted in a thread to maintain context in Open AI Assistants? API api , assistants-api , assistants-pricing	24	10383	January 29, 2024
How exactly do you get charged for using the API for assistants? API assistants-api	33	8196	November 27, 2023
Assistants API Pricing and Token Usage API api , pricing	103	35233	February 9, 2024
Assistants API pricing details per message API api-billing	67	42945	November 30, 2023
Token consumption: Prompt tokens exponentially increase when using Threads (Assistants) API assistants-api	8	1111	September 5, 2024

Reducing Context Tokens in Assistant Threads

Related topics