12

@Dent

I hope you see this. I’ve been looking around and I haven’t yet found a good solution for it.

Can i add more data to the model and then interact with it? For example: can i feed my entire Git repo into the model and then ask questions about the data? Can i give it an entire chat history and then ask it questions about it? I’ve seen embeddings being recommended but it still doesn’t fully address my requirements. Sure, i can give it a rolling chat history as part of the API request and it will “remember” some things but what I want is to put my data into the model itself and basically expand it’s “brain”.

Do you know of a way to do this?

1 Like
13

I solved the issue with a variable that stores previous messages and possible context from a file.
Reason i ended up to use files is that for example a power shortage may cut the conversation. This way its preserved in file. Ofcourse sql or some other solutions can be considered aswell…

In my program, i used the files for simplicity and mostly as proof-of-idea. You find it at github GitHub - EkBass/AiConsoleChat: OpenAi Chat Bot with some neat features

3 Likes
15

这也是我面临的一个大问题, 我正在努力解决. 找到解决方案将非常有帮助.

16

This is great. Thanks for posting. I’ve tried to implement similar using deque() to limit how many questions are “remembered”. The problem I’m facing is that ChatGPT will answer any previous questions every time I submit the string with history. Do you have a way to handle this behaviour?

17

When you send the data to the API, make sure the list elements are in order (system, user, assistant, user, assistant etc…) and that the last element in the list is the user prompt. If there are previous user prompts without an immediate corresponding assistant response, that specific unpaired user prompt will then receive the assistant answer.

-system: take on this behavior
-user: prompt goes here
-assistant: (the API response will be in response to the above)

-system: take on this behavior
-user: prompt goes here
-assistant: responds to the above user prompt
-user: prompt goes here
-assistant: (the API response will be in response to the above)

If you’re having trouble, ask daVinci or chatGPT.

1 Like
18

Yes… this happens if you use the completion API method and you send an array of prompts to the API using the prompt param.

If you want specific help, you need to post your code so we can see what are your API params and the specific API endpoint you are calling.

Thanks

:slight_smile:

1 Like
19

Thanks for the code, i’m using the same approach for this problem. And now i’m facing a new problem, its about the max_tokens that chatgpt can handle on their API request(i believe it’s only 4097 tokens maximum). How can i manage to keep this “memory” features now?

2 Likes
20

Would love to hear some feedback on this!

21

Thanks Dent. This is a nice little script to understand prompt engineering. I think there is a slight issue with your code snippet … I think that last line should read:
conversation_history = handle_input(user_input, conversation_history, USERNAME, AI_NAME) otherwise it does not retain the history …

Also, it is not clear to me whether there is any benefit in using the openai.ChatCompletion.create interface for this application and building up the messages parameter and adding roles like system, assistant, user.

1 Like
22

Will conversation history tokens charged for api usage or only new message considered for charging?

23

Of course, it will be charged. The more conversation there is, the more expensive each subsequent message will be.

1 Like
24

Doesn’t cover everything, but has a good overview of a full solution as does this video.

2 Likes
25
1 Like
26

To use the ChatGPT API without losing the continuity of the conversation, you will need to implement a persistent session management system on your end. Here’s how you can do it:

  1. When the user starts a conversation, send an API request to ChatGPT to initialize a session. Save the session ID that is returned by the API.
  2. Whenever the user sends a message, include the session ID along with the message text in the API request.
  3. When you receive a response from ChatGPT, save the session ID in a database or a cache along with the conversation history.
  4. If the user sends another message, use the saved session ID to resume the conversation by sending it along with the new message to the ChatGPT API.
  5. Repeat this process for each subsequent message in the conversation, always including the session ID with the message and saving the session ID and conversation history after each API call.

By implementing this persistent session management system, you can maintain the continuity of the conversation and ensure that the user’s context is preserved across multiple API requests.

2 Likes
27

I have created an alternative API for exactly this. You can use it to add conversation history to your GPT API calls and just use it like the GTP API with your own token.

Find it here: https://gptconverse.online/

28

This solution doesn’t seem very scalable. In terms of Big O, would this be considered O(n) time and space complexity? I was very surprised to find out this.

Does the actual ChatGPT website function the same way? Was the design decision here around privacy? And, lastly, is there any talk or roadmap about storing the message history on ChatGPT side?

29

You didn’t tell which “solution” you are referring to. You didn’t press the “reply” button on the relevant post.

This topic is quite old, starting even before the chat API was released, and has several diversions that are not answering “remember previous messages”.

Here’s a better recent thread to move to:

The conversation length cannot increase infinitely, because the AI model has a limited context length area to supply it past conversation. Management and truncation is required. The maximum compute would actually be in producing a very long output, as the input size has a smaller effect on the processing cost.

ChatGPT has very aggressive minimization of past conversation turns, giving the AI only what is needed to understand the present topic, so an otherwise ambiguous question like “what about the other one?” can be answered.

1 Like
30

OpenAI-Weaviate Query Process
OpenAI-Weaviate Query Process1609×1905 117 KB

This is a very basic process that has worked extremely well for me. I did a video on it here:

Based upon how it was explained to me in this conversation: Chat Completion Architechture

32

Is session id look like “chatcmpl-9IxJCCZAkqJVBZyk8FO12429sVwRc”?
Where should I put it in my request at step 4? OpenAI Platform - I can’t find any fields for session ID here.

33

The text quoted poorly describes what you’d do, the “poorly” starting at calling the API ChatGPT…

The ID returned with a response is just a unique identifier used internally by OpenAI for each API call return. It means nothing to you. If you log it along with your messages, in olden days of actually having support, you might say “my user 35185 made the request ID 385898634, but as I also sent the user name plus that call was first submitted to the moderations endpoint with id 385828531 returned, my account shouldn’t be banned”.

Examine ChatGPT. You have a hierarchy:

  • user name
  • user chats
  • user messages

Just have a database that records all those. Then for each new message the user would send, include the most recent messages of a chat that would fit in the token budget you have for chat history context to send to the API model.

1 Like