Assistants - Thread - Add Message vulnerable to incorrectly ordering messages

API
,
1

Working using the c# api - When I call

await openAi.Messages.CreateMessage(threadId, msgCreateRequest);

The message is created and the timestamp resolution in the createdAt only tracks to the nearest second. Eg: because the object tracks it using ‘ticks’ or ‘seconds’ as a unix-timestamp.

So the lowest resolution we can get for timestamp is as far as the nearest second.
Unfortunately this means that if we have 30 messages we want to programmatically add to a Thread, if we add them all in via a loop, without any Task.delay(1000) between each call - there is a very good chance that the messages will receive the same timestamp.

What consequence does this have for a user?
Well… it means that messages can be out of order.
If we are able to submit 3 messages within the window of a single second, then they will have the same timestamp on the server and the system won’t know how to order them.
If messages are out of order when the run starts - the assistant will have less ability to understand the context.

Further more - if a developer is writing a display for the messages in the thread, then when he pulls them down from OpenAI - they will similarly only have resolution down to seconds - which means that the developer needs some mechanism other than the createdAt field to track indexing.
Unfortunately the only work-around I can think of is to add an artificial 1000 ms delay between each message being added.

However, this means that if we are adding 1 second per message, and adding 30 messages, we essentially have to delay our run of the assistant by 1 second x number of messages = 30 second delay, just to have assurance that the messages are treated/considered in the correct order.

This could be resolved or mitigated by the Message object having the unix-timestamp shifted from an int32 (currently tracking to nearest second) to int64 (and then tracking to nearest millisecond)

It would end up being additional storage requirements in terms of data storage. However for a company the size of OpenAI these are probably negligible compared to the benefit provided.

A possible solution for the indexing issue for pulling messages down (for display etc) could be to use the metadata field and track the index as a simple number- then render the message in specific order once it is in c# land.
This doesn’t help us for running the assistant however.

For all I know - I am missing something. If anyone knows of another way of guaranteeing the order of messages, without relying on the createdAt field- which only has resolution to second’s - I am keen to hear a solution that doesn’t entail a 1 second delay between each message add. The delay will basically affect the performance of any consumer of the api.

2

I think your diagnosis here is wrong. You can only observe the symptom.

For example, add 30 messages in the messages parameter for “create thread”. They are not with a confused order.

It is more likely just the variable timeliness in individual API calls being committed and fulfilled.

1 Like
3

Your right - the issue is not with the create message method - the issue is that the only thing we have for ordering or indexing is the createdAt property/field - which has limited resolution - it only distinguishes between ‘seconds’.
In the context of a for-each loop - it is very likely that multiple messages, even if added sequentially - after one api call has finished before the next begins, these could still end up sharing the same ‘second’ interval in terms of a unix-timestamp.

Maybe openAI has something ‘else’ server side that maintains ordering or indexing - however, this is lost once the messages are retrieved by a client/user - and if someone was to perform an OrderBy - using the createdAt property - there would be no guarantee that the messages would be in the correct order.

Similarly - the ListMessages call- with pagination, indicates that it uses the createdAt property - either asc or desc order - but it does rely on it - so it is unclear whether 3 messages sharing the same timestamp would order correctly.

The first time I observed this issue - was me just inserting 2 messages into a thread.

                msgIdStart = await openAI.AddMessageToThread(message, this.OAI_ThreadId);
				await Task.Delay(1000);
                string newMsg = await openAI.AddMessageToThread(content, this.OAI_ThreadId);

These two messages - if done without the Task.Delay(1000) - would be retrieved at a later date by my code - and there would no guarantee they would be in the right order.
I had to add a Task.Delay call - to guarantee they would have a different unix-timestamp.
This results in a 1 second artificial delay - for the sake of preserving the message ordering.
In the situation of adding a larger amount of messages - a one second delay is required between each message to ensure they have unique incremental unix timestamps.

Right now the way I am working around this is to use the metadata field - to add the timestamp with millisecond resolution - but it is putting more onus on the user of the API to create a reliable ordering system as a work-around.

4

When retrieving at a later date - to try and maintain order - I am using the below:

//Original sorting code
allMessages = allMessages.OrderBy(e => e.CreatedAt).ToList();

//Conditionally sort based on metadata present field.
allMessages = allMessages.OrderBy(e =>
            {
                // If metadata contains "timestamp", parse it as Ticks and convert to DateTime
                if (e.Metadata.TryGetValue("timestamp", out string ticksString) && long.TryParse(ticksString, out long ticks))
                {
                    return new DateTime(ticks, DateTimeKind.Utc); // Convert Ticks to DateTime
                }

                // Otherwise, use CreatedAt (Unix seconds) converted to DateTime
                return DateTimeOffset.FromUnixTimeSeconds(e.CreatedAt).UtcDateTime;
            }).ToList();

This however relies on the developer/user including a tick’s to the message metadata when they are adding the message to the thread - otherwise we are having to rely on the existing ‘second’ level time-resolution. Resolving down to the second is better than nothing at all, but if the millisecond level resolution is present - try to use it instead.

public async Task<string> AddMessageToThread(string msg,string threadId)
        {
            var openAi = GetClient();

            var msgCreateRequest = new MessageCreateRequest() { Content = new OpenAI.ObjectModels.MessageContentOneOfType(msg) };
            msgCreateRequest.Metadata.Add("timestamp", DateTime.Now.Ticks.ToString());
            var resp = await openAi.Messages.CreateMessage(threadId, msgCreateRequest);

            if (!resp.Successful)
            {
                return null;
            }

            return resp.Id;
        }
5

I still don’t think I dissuaded you from your misconception:

  • The “created” timestamps are just informational for you.
  • The “created” timestamps do not have a bearing on “sorting.”
  • In a thread, messages are positional.
  • In thread messages, there’s going to be a relational field, like "last: “msg_id_8355” if not flat file, which maintains internal order.
  • There is no sending time information to the API that will change the ordering in which messages are “played” to the AI.

Your symptom completely arises because time in transit, being routed to different servers within OpenAI, variable HTTPS latency, internal server queue depth, etc., can mean that API packets or assembled API calls are processed out of order from how your code iterated through them.

You have figured out:

  • Sending many messages at once when you create a thread is available.
  • Checking that a message has been successfully added to the thread list before another is appended is required.

What is hard to figure out:

Why you are sending 20 messages, but not with a method that supports a full list. Why you are not sending them all at once as a single Chat.Completions API call. If you are the one maintaining a conversation history and not OpenAI, that nullifies the whole point of Assistant’s server-side threads.

6

Its not a misconception that openai pagination is derived / based on the createdAt timestamp - this is directly stated in their api reference.

https://platform.openai.com/docs/api-reference/messages/listMessages

order

string

Optional

Defaults to desc

Sort order by the created_at timestamp of the objects. asc for ascending order and desc for descending order.

You have alluded to the existence of a relational field that provides the internal order, which might very well exist - but the api reference does not expose this and it certainly isn’t provided to the consumer/client for client side sorting. It is a reasonable assumption to make that if the ListMessages endpoint returns a List of messages, someone might want to process or sort/order that list. If it was returned as a relational or linked list of some sort - that might be useful… but I don’t think it is. Its returned as a simple array.

I’ve confirmed that the ListMessages endpoint does ‘seem’ to return the elements in the right order in the following experiment, so maybe there is that internal sorting field on the server - however, because the only field we have for client side sorting is the createdAt field - there is nothing else available for use - as soon as someone does a sort - it does often result in out of sequence elements when 2 or more share the same timestamp. I’ve replicated it with this code.

Note - these are using the await operator which ensures the start and end of each api operation. This is tcp / http - not udp fire and forget - if the await operator does not correctly indicate the completion of an API endpoint call, then it would be chaos. The whole point of the await operator is to wait for the completion of the call. AddMessage is not the same as CreateRun - where it spawns an external process which we have to monitor via further calls. As soon as the method call is completed - the message is created in the thread and it has the final createdAt value and the final text content - we get the message id back from each call.

await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);
                await openAI.AddMessageToThread(DateTime.Now.Ticks.ToString(), this.OAI_ThreadId);

                var allMessagesNew = await this.GetMessages();
                allMessagesNew.Data = allMessagesNew.Data.OrderBy(e => e.CreatedAt).ToList();

                var first7 = allMessagesNew.Data.TakeLast(21);
                foreach(var msg in first7)
                {
                    Console.WriteLine($"{msg.CreatedAt} -> {msg.Content.First().Text.Value}");
                }

Without
allMessagesNew.Data = allMessagesNew.Data.OrderBy(e => e.CreatedAt).ToList();

The messages are in sequence as soon as they are received by the client, however as soon as client-side code tries to do any form of meaningful sorting - we get out of sequence elements - in fact, once the messages are out of sequence, we have nothing we can orderby or sort by that could possibly get them back into the correct order - we would need to request them from the server again. Correct me if I am wrong - is there something the client can use to sort on besides CreatedAt ?

ChatGPT helped with the analysis of these to find out of sequence elements.
Basically any where ‘True’ is present - indicates it was out of sequence with the one that preceded it - arguably the one before it should be ‘true’ as well - but blame chatgpt for that.

,CreatedAt (Unix Seconds),Metadata Timestamp (Ticks),Out of Sequence
0,2025-02-16 00:48:55+00:00,2025-02-16 00:48:55.686698+00:00,False
1,2025-02-16 00:48:56+00:00,2025-02-16 00:48:56.488032+00:00,False
2,2025-02-16 00:48:56+00:00,2025-02-16 00:48:56.078016+00:00,True
3,2025-02-16 00:48:57+00:00,2025-02-16 00:48:57.715402+00:00,False
4,2025-02-16 00:48:57+00:00,2025-02-16 00:48:57.217054+00:00,True
5,2025-02-16 00:48:57+00:00,2025-02-16 00:48:56.896362+00:00,True
6,2025-02-16 00:48:58+00:00,2025-02-16 00:48:58.535435+00:00,False
7,2025-02-16 00:48:58+00:00,2025-02-16 00:48:58.124789+00:00,True
8,2025-02-16 00:49:00+00:00,2025-02-16 00:48:58.944913+00:00,False
10,2025-02-16 00:49:01+00:00,2025-02-16 00:49:01.300892+00:00,False
9,2025-02-16 00:49:01+00:00,2025-02-16 00:49:01.709664+00:00,False
11,2025-02-16 00:49:02+00:00,2025-02-16 00:49:02.426264+00:00,False
12,2025-02-16 00:49:02+00:00,2025-02-16 00:49:02.119255+00:00,True
13,2025-02-16 00:49:03+00:00,2025-02-16 00:49:03.559056+00:00,False
14,2025-02-16 00:49:03+00:00,2025-02-16 00:49:03.149332+00:00,True
15,2025-02-16 00:49:03+00:00,2025-02-16 00:49:02.835359+00:00,True
16,2025-02-16 00:49:04+00:00,2025-02-16 00:49:04.577891+00:00,False
17,2025-02-16 00:49:04+00:00,2025-02-16 00:49:04.269784+00:00,True
18,2025-02-16 00:49:04+00:00,2025-02-16 00:49:03.861926+00:00,True
19,2025-02-16 00:49:05+00:00,2025-02-16 00:49:05.193953+00:00,False
20,2025-02-16 00:49:05+00:00,2025-02-16 00:49:04.894189+00:00,True

I will concede - if I rely on the ordering provided by ListMessages - there is a good chance it will result in correct order so far. However I do need to do processing client side - which seems to be vulnerable to the lack of reliable indexing field. I suppose I could modify the message elements after they are received from the ListMessages endpoint, to preserve their index in the metadata - and then doing sorting on that metadata field. This is merely a work around.

You have asked why I am doing bulk additions to a thread.

For context - in my project - I need to provide contextual information to the assistant which helps to shape their response, but I do not want to store that persistently in the thread - because I don’t want it to contaminate the displayed text the user sees or expose it to the assistant in future runs. Eg: it contains miscellaneous context information the user shouldn’t see, but the assistant needs for ‘that’ run.

I call these ephemeral messages - I used to delete these messages after the run was completed. However it was risky to add these messages to the main thread because a system error or interruption could result in the messages being left inside the main thread and never cleaned out.
An example for an ephemeral message is to remind the assistant to not do certain things or to format the data in specific ways. These reminder messages are unsightly and do not need to be repeated multiple times- the place where they are most valuable is right at the end of the message chain - so we add it in at the end, run the thread, then delete the messages that we don’t care about. This is how I ‘used’ to do it.

The new approach I am taking is to spawn a completely different thread and provide a sub-set of the data from the main thread and then run on that thread - then just discard the thread at the end as a throw away thread - then transfer the information needed to the main thread. I add the ephemeral messages, but I am less concerned about clean up - because I am discarding the whole thread.

At the end of the run on the throw away thread, I usually extract two messages from it and then transplant them into the main thread.

The two messages I usually end up adding to the main thread end up being the ‘user’ message that the triggered the interaction (with some modifications), and the ‘response’ message that the assistant provided from the throw away thread.
If I add these to the main thread without an artificial delay - it results in a good chance for timestamp duplication. If these get the same createdAt timestamp, then any further calls I make to orderBy(e=> e.CreatedAt) - would run the risk of getting duplicate timestamp entries out of order.

So while it is not out of order when it comes from the server, its definitely a risk when sorting on the client - and there is nothing that the api exposes to provide reliable ordering on client side - unless I’ve missed that in the documentation somewhere.