Message OR Response? A more viable solution for your database

Community
1

Hi, I have been seeing that everyone has been doing chats like this:

Screen Shot 2023-11-11 at 11.24.37
Screen Shot 2023-11-11 at 11.24.37764×602 27.8 KB

Wouldn’t it make more sense to do something like this:

Screen Shot 2023-11-11 at 11.26.52
Screen Shot 2023-11-11 at 11.26.52766×599 28.3 KB

Doing this has several advantages… for example, if you need to analyze user messages, it is easier to see the input and output, the output would be the AI’s response. Likewise, in the UI you can add the response message and only update the output every time the stream changes, this way you would no longer have the user and ai messages separately.

I did it this way and it served me much better, every time a user sends an input, I append a new response to my messages with the input and the output null, when the function is completed I update only the output.

Are there any disadvantages to doing it this way?

This obviously wouldn’t work in a live chat like WhatsApp, but since AI chats call a function that contains the input logic as well as the output, it made it more logical to store everything in my database under the same row.

What do you think?

Prisma Model:

// I called the message => response...
model response {
  id           String   @id @unique @default(cuid())
  createdAt    DateTime @default(now()) @map(name: "created_at")
  input        String
  output       String
  embeddingIds String[] @map(name: "embedding_ids")

  // tokens
  inputTokens     Int @map(name: "input_tokens")
}

Then:

message {
  id: "...."
  content: "How are you?"
  role: "user"
}

message {
  id: "...."
  content: "Im an ai..."
  role: "assistant"
}

Now:

response {
  id: "...."
  input: "How are you?"
  output: "Im an ai..."
}
1 Like
2

The disadvantage in saving pairs is that they are state dependent.

It’s much easier to save each thing separately with a a timestamp and sort it out in software IMO.

Also what do you do for the non-pair case. So assistant/assistant/user or user/user/assistant. This easily breaks the pair assumption.

1 Like
4

Hi, What do you mean by “state dependent”?

5

You are combining two things as (input, output) pairs. So to create this pair, you track the input event, then wait for an output event to be emitted, then create the new (input, output) vector pair state. So you are holding (delaying) the input message until a state change of an emitted output.

Try actually coding this, and see that it is more work than just time stamping and retrieving events from the database in time order.

If this pairing is important, you can quickly and easily synthesize it directly out of your database.

6

I already made it, it seems to work fine… the user sees the input immediately in the ui, and the output is updated while the response streams, when the stream success then you add both the input and output in the database. In case the output fails you just add an empty output. So basically the user will see all like normal, but the database will only receive the new row once the stream is successful.

7

I have no doubt that it works. But you were asking why doesn’t everyone do this. And basically, the answer is it is more work.

To make your system more scalable and event driven, you want separation of concerns and stateless.

Just experience talking here :rofl:

3 Likes
8

Thank you! I just wanted to know if it has disadvantages, I think your answer helped me.

3 Likes