Hello guys, I have a simple question, how the official Chat-GPT website remember the previous messages and when I use the API it doesn’t remember it. For example I said to GPT-3.5-turbo “my name is Abdessattar” and after a while I will ask him again “What is my name” it will answer me “I’m sorry, as an AI language model, I cannot recall previous interactions or conversations as I do not have the capability to store data.”
So, is there any solution.
4 Likes
You have to send the entire conversation back to the API each time. ChatGPT does not remember conversations, it is just sending what is in the chat window back to the API every time you hit submit.
9 Likes
can you give me more information like how to do it (I am using there nodejs library) Please.
I can give you a simple Python example:
That should help you see what it’s doing. You basically need to store the chat history yourself and then send it up each time.
Thank you, but I acctualy work with nodejs.
1 Like
go to this link, OpenAI API
look a the example
# Note: you need to be using OpenAI Python v0.27.0 for the code below to work
import openai
openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Who won the world series in 2020?"},
{"role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020."},
{"role": "user", "content": "Where was it played?"}
]
)
what you need to do in your app is add to the messages array in each interaction, for example:
when a user Joins the chat and says hello
# Note: you need to be using OpenAI Python v0.27.0 for the code below to work
import openai
openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello"},
]
)
When OpenAI replies, you add that to your messages array
# Note: you need to be using OpenAI Python v0.27.0 for the code below to work
import openai
openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello"},
{"role": "assistant", "content": "Hello, how can I help you?"},
]
)
when the user enters more text you send the whole array back to the server
# Note: you need to be using OpenAI Python v0.27.0 for the code below to work
import openai
openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello"},
{"role": "assistant", "content": "Hello, how can I help you?"},
{"role": "user", "content": "who is more stylish Pikachu or Neo"},
]
)
then you keep adding each interaction to the messages array as you send it back to the server.
# Note: you need to be using OpenAI Python v0.27.0 for the code below to work
import openai
openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello"},
{"role": "assistant", "content": "Hello, how can I help you"},
{"role": "user", "content": "who is more stylish Pikachu or Neo"},
{"role": "assistant", "content": "Well Neo of course"},
]
)
and continue
# Note: you need to be using OpenAI Python v0.27.0 for the code below to work
import openai
openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello"},
{"role": "assistant", "content": "Hello, how can I help you"},
{"role": "user", "content": "What is more stylish Pikachu or Neo"},
{"role": "assistant", "content": "Well Neo of course"},
{"role": "user", "content": "Why?"},
]
)
when the user says “Why?” the only way for ChatGPT to know what you are talking about is by sending the entire conversation, this is how it gets context.
# Note: you need to be using OpenAI Python v0.27.0 for the code below to work
import openai
openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello"},
{"role": "assistant", "content": "Hello, how can I help you"},
{"role": "user", "content": "What is more stylish Pikachu or Neo"},
{"role": "assistant", "content": "Well Neo of course"},
{"role": "user", "content": "Why?"},
{"role": "assistant", "content": "Well Pikachu is naked! there is no style in that"},
]
)
In your application, you need to have a data structure to represent conversations and that data structure needs to be able to accept an array of messages, and be able to be updated with new messages coming from either the user or the server, then whenever the user updates the data structure you send it to the server whenever you get a reply from the server you update the data structure and present it in the UI.
7 Likes
Thanks for the helpful example.
Does this mean the context window gets used up quickly, at which point chatGPT can no longer continue the conversation with full recall/ memory? Are there ways around this?
1 Like
Yes, at some point you will run out of tokens for the history and you’ll need to throw away some of the older messages.
There are a few ways to work around this - one thing you can do is ask it to summarise the conversation so far and keep that in the history so it has something to work from.
7 Likes
Yes, at some point you will run out of tokens for the history and you’ll need to throw away some of the older messages.
Yes, and,
Watch out for the escalating use or resources. Generally the api, especially completions, queries cost fractions of a penny. When you get a few interactions going with chat the cost escalates rapidly
Keep an eye on the return headers for the number of tokens used.
I have gotten I to the high fractions of a dollar after about a dozen rounds
3 Likes
if only there was some sort of magical site that could transform code in one language into another, and then give a nice human readable explanation of how the code works.
3 Likes
Thanks for the longer, very explicit example. Much appreciated!
yo i am having the same problem, i am providing the context from some external documentation using vector embeddings and adding the context as {“role”: “user”, “content”:“text extracted using vector embeddings”}
but is not remembering the context though i am appending all the message to the messsages list, here’s the conversation for the terminal bot i am creating for more details:
(.venv) yug@Yugs-MacBook-Air pgvector-implementation % python3 main.py
User: hello
Messages sent to the API: [{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'user', 'content': "(' Intercom\\n',)"}, {'role': 'user', 'content': 'Question: hello'}]
GPT: Hello! How can I assist you today?
User: my name is yug, i have a an online dukaan store, i want to keep it closed this tuesday. how to do that?
Messages sent to the API: [{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'user', 'content': "(' Intercom\\n',)"}, {'role': 'user', 'content': 'Question: hello'}, {'role': 'assistant', 'content': 'Hello! How can I assist you today?'}, {'role': 'user', 'content': "('What should I do if I do not want to receive orders on a particular day? | Dukaan® Help CenterGo to DukaanGo to DukaanAll CollectionsStore ProfileWhat should I do if I do not want to receive orders on a particular day?What should I do if I do not want to receive orders on a particular day?Learn how to switch your store Offline or OnlineNWritten by Namrata Sawhney Updated over a week agoJust like physical stores are closed at the end of the day or not opened due to any reason, you can also switch your Dukaan store Offline or Online as per your convenience. To turn your store Offline, Click on the toggle button at the top right which says Online. Choose the option when you want your store to be Online again. Your store will automatically be Online, once the selected time frame is over. Once your store is online, your customers can again place the orders. Only if you choose I’ll go online myself option, you have to manually switch your store to Online, whenever you want to. Click Confirm and your store will be Offline. To switch your store online at any time, click on the toggle that says Offline at the top right to turn it back to Online. Did this answer your question?😞😐😃Dukaan® for PCPrivacy PolicyAbout Dukaan®FacebookTwitterLinkedInIntercomWe run on Intercom\\n',)"}, {'role': 'user', 'content': 'Question: my name is yug, i have a an online dukaan store, i want to keep it closed this tuesday. how to do that?'}]
GPT: Hi Yug! To keep your Dukaan store closed on a particular day, you can follow these steps:
1. Open your Dukaan app and go to your store profile.
2. Look for the toggle button at the top right corner, which says "Online".
3. Click on the toggle button to switch your store from Online to Offline.
4. After clicking on the toggle button, you will see an option to choose the time frame for which you want your store to remain offline.
5. Choose the date and time when you want your store to go online again.
6. Click on the "Confirm" button to switch your store to offline mode.
Once you have switched your store to offline mode, your customers will not be able to place any orders until your store goes online again as per the time frame you have chosen. You can switch your store to online mode by clicking the same toggle button at any time.
User: what is my name?
Messages sent to the API: [{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'user', 'content': "(' Intercom\\n',)"}, {'role': 'user', 'content': 'Question: hello'}, {'role': 'assistant', 'content': 'Hello! How can I assist you today?'}, {'role': 'user', 'content': "('What should I do if I do not want to receive orders on a particular day? | Dukaan® Help CenterGo to DukaanGo to DukaanAll CollectionsStore ProfileWhat should I do if I do not want to receive orders on a particular day?What should I do if I do not want to receive orders on a particular day?Learn how to switch your store Offline or OnlineNWritten by Namrata Sawhney Updated over a week agoJust like physical stores are closed at the end of the day or not opened due to any reason, you can also switch your Dukaan store Offline or Online as per your convenience. To turn your store Offline, Click on the toggle button at the top right which says Online. Choose the option when you want your store to be Online again. Your store will automatically be Online, once the selected time frame is over. Once your store is online, your customers can again place the orders. Only if you choose I’ll go online myself option, you have to manually switch your store to Online, whenever you want to. Click Confirm and your store will be Offline. To switch your store online at any time, click on the toggle that says Offline at the top right to turn it back to Online. Did this answer your question?😞😐😃Dukaan® for PCPrivacy PolicyAbout Dukaan®FacebookTwitterLinkedInIntercomWe run on Intercom\\n',)"}, {'role': 'user', 'content': 'Question: my name is yug, i have a an online dukaan store, i want to keep it closed this tuesday. how to do that?'}, {'role': 'assistant', 'content': 'Hi Yug! To keep your Dukaan store closed on a particular day, you can follow these steps:\n\n1. Open your Dukaan app and go to your store profile.\n2. Look for the toggle button at the top right corner, which says "Online".\n3. Click on the toggle button to switch your store from Online to Offline.\n4. After clicking on the toggle button, you will see an option to choose the time frame for which you want your store to remain offline.\n5. Choose the date and time when you want your store to go online again.\n6. Click on the "Confirm" button to switch your store to offline mode.\n\nOnce you have switched your store to offline mode, your customers will not be able to place any orders until your store goes online again as per the time frame you have chosen. You can switch your store to online mode by clicking the same toggle button at any time.'}, {'role': 'user', 'content': "('How to show my store name on my website? | Dukaan® Help CenterGo to DukaanGo to DukaanAll CollectionsStore ProfileHow to show my store name on my website?How to show my store name on my website?Learn how to show your store name on your eCommerce website using DukaanNWritten by Namrata Sawhney Updated over a week agoThe preferences section on web allows you to show your store name alongside the store logo at the top of the navigation bar. To access the preferences section, click on the dropdown in the top right corner. Now, click on Store Settings. Select Preferences from the panel on the left. The show store name toggle in the header section allows you to add your store name alongside the logo on top of the navigation bar. By default, this feature is turned on. This is how your store will look like to the buyer when the toggle is switched on. This is how your store will look like to the buyer when the toggle is switched off. To enable this feature, set the show store name toggle to on by clicking on it.Read MoreHow to use Advance Catalogue?How to Auto Accept Orders?How to set order preferences?Did this answer your question?😞😐😃Dukaan® for PCPrivacy PolicyAbout Dukaan®FacebookTwitterLinkedInIntercomWe run on Intercom\\n',)"}, {'role': 'user', 'content': 'Question: what is my name?'}]
GPT: I apologize, but as an AI language model, I don't have access to personal information such as your name. Could you please tell me your name? I'll be happy to assist you with any questions or concerns you have.
User: i told you previosly, look in the history
Messages sent to the API: [{'role': 'system', 'content': 'You are a helpful assistant.'}, {'role': 'user', 'content': "(' Intercom\\n',)"}, {'role': 'user', 'content': 'Question: hello'}, {'role': 'assistant', 'content': 'Hello! How can I assist you today?'}, {'role': 'user', 'content': "('What should I do if I do not want to receive orders on a particular day? | Dukaan® Help CenterGo to DukaanGo to DukaanAll CollectionsStore ProfileWhat should I do if I do not want to receive orders on a particular day?What should I do if I do not want to receive orders on a particular day?Learn how to switch your store Offline or OnlineNWritten by Namrata Sawhney Updated over a week agoJust like physical stores are closed at the end of the day or not opened due to any reason, you can also switch your Dukaan store Offline or Online as per your convenience. To turn your store Offline, Click on the toggle button at the top right which says Online. Choose the option when you want your store to be Online again. Your store will automatically be Online, once the selected time frame is over. Once your store is online, your customers can again place the orders. Only if you choose I’ll go online myself option, you have to manually switch your store to Online, whenever you want to. Click Confirm and your store will be Offline. To switch your store online at any time, click on the toggle that says Offline at the top right to turn it back to Online. Did this answer your question?😞😐😃Dukaan® for PCPrivacy PolicyAbout Dukaan®FacebookTwitterLinkedInIntercomWe run on Intercom\\n',)"}, {'role': 'user', 'content': 'Question: my name is yug, i have a an online dukaan store, i want to keep it closed this tuesday. how to do that?'}, {'role': 'assistant', 'content': 'Hi Yug! To keep your Dukaan store closed on a particular day, you can follow these steps:\n\n1. Open your Dukaan app and go to your store profile.\n2. Look for the toggle button at the top right corner, which says "Online".\n3. Click on the toggle button to switch your store from Online to Offline.\n4. After clicking on the toggle button, you will see an option to choose the time frame for which you want your store to remain offline.\n5. Choose the date and time when you want your store to go online again.\n6. Click on the "Confirm" button to switch your store to offline mode.\n\nOnce you have switched your store to offline mode, your customers will not be able to place any orders until your store goes online again as per the time frame you have chosen. You can switch your store to online mode by clicking the same toggle button at any time.'}, {'role': 'user', 'content': "('How to show my store name on my website? | Dukaan® Help CenterGo to DukaanGo to DukaanAll CollectionsStore ProfileHow to show my store name on my website?How to show my store name on my website?Learn how to show your store name on your eCommerce website using DukaanNWritten by Namrata Sawhney Updated over a week agoThe preferences section on web allows you to show your store name alongside the store logo at the top of the navigation bar. To access the preferences section, click on the dropdown in the top right corner. Now, click on Store Settings. Select Preferences from the panel on the left. The show store name toggle in the header section allows you to add your store name alongside the logo on top of the navigation bar. By default, this feature is turned on. This is how your store will look like to the buyer when the toggle is switched on. This is how your store will look like to the buyer when the toggle is switched off. To enable this feature, set the show store name toggle to on by clicking on it.Read MoreHow to use Advance Catalogue?How to Auto Accept Orders?How to set order preferences?Did this answer your question?😞😐😃Dukaan® for PCPrivacy PolicyAbout Dukaan®FacebookTwitterLinkedInIntercomWe run on Intercom\\n',)"}, {'role': 'user', 'content': 'Question: what is my name?'}, {'role': 'assistant', 'content': "I apologize, but as an AI language model, I don't have access to personal information such as your name. Could you please tell me your name? I'll be happy to assist you with any questions or concerns you have."}, {'role': 'user', 'content': "(' open it, you can see a black navigation bar on the top. This navigation bar displays all the items be it products, categories, pages, or custom links here. As described above, we had added a product, category, and About Us page to the navigation bar which can be seen here. Also, we nested the product under the category to create a drop-down menu. To see it, hover over the category name and you can see the products listed under it. Did this answer your question?😞😐😃Dukaan® for PCPrivacy PolicyAbout Dukaan®FacebookTwitterLinkedInIntercomWe run on Intercom\\n',)"}, {'role': 'user', 'content': 'Question: i told you previosly, look in the history'}]
GPT: I apologize if I missed your previous query. Can you please repeat your question or concern so that I can assist you?
User:
I used the code below for an Express app. However, it throws an error around the messages property in the openai.createCompletion function. Does anyone have an idea?
let conversation = [
{
role: 'system',
content: 'You will follow the conversation and respond to the queries asked by the \'user\'\'s content. You will act as the assistant'
}
];
app.post('/', async(req, res) => {
try {
const prompt = req.body.prompt;
conversation.push({
role: 'user',
content: prompt
})
const response = await openai.createCompletion({
model: 'text-davinci-003',
// messages: conversation.map(({ role, content }) => ({ role, content })),
messages: conversation
});
conversation.push({
role: 'assistant',
content: response.data.choices[0].text
})
res.status(200).send({
bot: response.data.choices[0].text
});
} catch (error) {
console.error(error)
res.status(500).send(error || 'Something went wrong');
}
})
I believe text-davinci-003 uses “prompt” instead of role/content messages.
Maybe try adjust the model to gpt-3.5-turbo for the code. 
I modified my code to the one below but still get this error (Error: Request failed with status code 404)
Code:
let conversation = [
{
role: 'system',
content: "You will follow the conversation and respond to the queries asked by the 'user's content. You will act as the assistant"
}
];
app.post('/chat', async (req, res) => {
try {
// prompt+='user: ' +req.body.prompt + '\n'
let prompt = req.body.prompt
conversation.push(
{
role: 'user',
content: prompt
}
)
const response = await openai.createCompletion({
model: "gpt-3.5-turbo",
messages: conversation,
max_tokens:500
});
// prompt += 'assistant: ' + response.data.choices[0].text.trim() + '\n\n'
// console.log(prompt)
conversation.push(
{
role: 'assistant',
content: response.data.choices[0].text
})
res.status(200).send({
bot: response.data.choices[0].text
});
} catch (error) {
console.error(error)
res.status(500).send(error || 'Something went wrong);
}
})
1 Like
The API is stateless which means you have to send all your previous messages that you want used for the chat history with each new request
This is a great relevant topic because I see little point in not keeping context. I mulled about doing it myself but then I found a great tutorial made by Voiceflow that also features Alexa integration. It basically boils down to Javascript arrays and appending replies to the initial array. Something like:
messages = []
messages.push...
...
messages.push({"role": "user", "content": user_reply})
...
response.choices[0].message.content
=> {gpt_reply}
...
messages.push({"role": "assistant", "content": gpt_reply})
...
I’m blocked from adding links to this forum but if you go to Voiceflow’s blog and look in the Developers blog category, the article is titled: “How to create an Alexa skill with GPT-4 and Voiceflow”
@rbritom example is valid but what about the long conversation? If I pass the whole message then I’ll get a token limitation error what can I do?
What to do about long conversations? That has many answers with progressive quality of memory, where we first calculate and store the tokens used by each user input and ai response (and calculate the overhead of the chat format):
- Discard older conversation turns that won’t fit into the remaining context space after calculating tokens locally for prompt, user input, max_tokens reservation.
- Use another AI to periodically summarize turns or the oldest conversation.
- Use a vector database and AI embeddings to remember the whole conversation and pass prior exchanges that are most relevant to the current conversation flow.
- More advanced context-aware conversation thread topic tracking systems.
Another option for the advanced user is a GUI that allows self-management of the conversation and selection of past turns to be sent.
4 Likes
I have found the solution you need to calculate the token before passing it to the openai API,
Tiktoken helps me a lot you can find tokens of messages list of any language, I have added code below please check. you need to update the messages list, for example, if your token limit exceeds so you can remove the very first messages to full fill the token limit error.
If you still have any questions or face any issues feel free to contact me on Linkedin: aliahmadjakhar
def num_tokens_from_messages(messages, model="gpt-3.5-turbo-0613"):
"""Return the number of tokens used by a list of messages."""
try:
encoding = tiktoken.encoding_for_model(model)
except KeyError:
print("Warning: model not found. Using cl100k_base encoding.")
encoding = tiktoken.get_encoding("cl100k_base")
if model in {
"gpt-3.5-turbo-0613",
"gpt-3.5-turbo-16k-0613",
"gpt-4-0314",
"gpt-4-32k-0314",
"gpt-4-0613",
"gpt-4-32k-0613",
}:
tokens_per_message = 3
tokens_per_name = 1
elif model == "gpt-3.5-turbo-0301":
tokens_per_message = 4 # every message follows <|start|>{role/name}\n{content}<|end|>\n
tokens_per_name = -1 # if there's a name, the role is omitted
elif "gpt-3.5-turbo" in model:
print("Warning: gpt-3.5-turbo may update over time. Returning num tokens assuming gpt-3.5-turbo-0613.")
return num_tokens_from_messages(messages, model="gpt-3.5-turbo-0613")
elif "gpt-4" in model:
print("Warning: gpt-4 may update over time. Returning num tokens assuming gpt-4-0613.")
return num_tokens_from_messages(messages, model="gpt-4-0613")
else:
raise NotImplementedError(
f"""num_tokens_from_messages() is not implemented for model {model}. See https://github.com/openai/openai-python/blob/main/chatml.md for information on how messages are converted to tokens."""
)
num_tokens = 0
for message in messages:
num_tokens += tokens_per_message
for key, value in message.items():
num_tokens += len(encoding.encode(value))
if key == "name":
num_tokens += tokens_per_name
num_tokens += 3 # every reply is primed with <|start|>assistant<|message|>
return num_tokens
1 Like