Use openai.createChatCompletion it’s simple 
Is openai.createChatCompletion has no token limit? what happed is the token limit exceeded?
I developed this node.js script to save every GPT-4 request and response into a JSON file.
It then sends the file content as conversation history with each new prompt.
// Import required modules
const fs = require('fs');
const axios = require('axios');
// Your OpenAI API key
const apiKey = 'your-openai-api-key';
// Function to interact with OpenAI API
async function interactWithAI(userPrompt) {
try {
// Define the message data structure
let messageData = { 'messages': [] };
// If requests.json exists, read and parse the file
if (fs.existsSync('requests.json')) {
let raw = fs.readFileSync('requests.json');
messageData = JSON.parse(raw);
}
// Format the conversation history and the new user request
let systemMessage = "Conversation history:\n" + messageData['messages'].map(m => `${m.role} [${m.timestamp}]: ${m.content}`).join("\n");
let userMessage = "New request: " + userPrompt;
// Make a POST request to OpenAI's chat API
let response = await axios({
method: 'post',
url: 'https://api.openai.com/v1/chat/completions',
headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' },
data: { 'model': 'gpt-4', 'messages': [ { "role": "system", "content": systemMessage }, { "role": "user", "content": userMessage } ] }
});
// Log the AI's response
console.log(response.data['choices'][0]['message']['content']);
// Get the current timestamp
let timestamp = new Date().toISOString();
// Add the new user request and the AI's response to the message history
messageData['messages'].push({ "role": "user", "content": userPrompt, "timestamp": timestamp });
messageData['messages'].push({ "role": "assistant", "content": response.data['choices'][0]['message']['content'], "timestamp": timestamp });
// Write the updated message history to requests.json
fs.writeFileSync('requests.json', JSON.stringify(messageData, null, 2));
// Return the AI's response
return response.data['choices'][0]['message']['content'];
} catch (e) {
// If an error occurred, log it to the console and return an error message
console.error('An error occurred:', e);
return 'An error occurred while interacting with the OpenAI API. Please check the console for more details.';
}
}
cc: @abdeldroid
1 Like
But when I try CHATGPT, I ask same question twice with a bit different tones, 1st time it replies an answer and add “…please notice this is only a fairly answer, blah blah blah”, and 2nd time it answers again but this time it says “Again I have to remind you, blah blah blah”. It seems to me that CHATGPT knows I ask same question twice. How does it do this?
This discussion is about programming the API, where the programmer must construct each new call to the API AI model by including their own “memory” of what was previously said.
In ChatGPT (the website chatbot you can talk with), this conversation history database and the management of what prior conversation the AI sees when it answers is handled by the ChatGPT web and server software and its programmers.
2 Likes
Oh! I see. So I have to redesign whole chatting ability myself when use API. That’s really a challenge to me.
1 Like
I wrote up a very small python gpt-3.5-turbo chatbot for you:
- Complete record of conversation,
"chat", - passes 5 previous question/answers from chat to the AI so the topic is clear,
- streaming generator, so you receive words as they are created,
- tolerates no API errors
- edit in your own API key to use; type exit to leave
import openai
openai.api_key = "sk-xxxxx"
system = [{"role": "system", "content": "You are a helpful AI assistant."}]
user = [{"role": "user", "content": "Introduce yourself."}]
chat = []
while not user[0]['content'] == "exit":
response = openai.ChatCompletion.create(
messages = system + chat[-10:] + user,
model="gpt-3.5-turbo", stream=True)
reply = ""
for delta in response:
if not delta['choices'][0]['finish_reason']:
word = delta['choices'][0]['delta']['content']
reply += word
print(word, end ="")
chat += user + [{"role": "assistant", "content": reply}]
user = [{"role": "user", "content": input("\nPrompt: ")}]
(revised: the AI provides an introduction so we immediately check connection, and one less conditional.)
The openai library requires python 3.7-3.9. Install the module with
pip install --upgrade openai
The next level is to use a token counter (tiktoken), record the size of each chat message, and use some size criteria to determine how many past messages to send.
1 Like
To be clear, the messages are just injected directly in-prompt, correct? OpenAI doesn’t do any fancy summarization of previous messages or embedding/retrieval like they do on chatgpt.openai.com, correct?
You provide the past conversation yourself by your code. A more advanced chatbot can make occasional summaries, can use a vector database to retrieve relevant chats that are even older, can continue increasing the number of turns up to the context length limit, or instead intelligently cut chat to minimum, and more.
What my code does is the typical basic operation:
- always pass the system instruction unaltered as the first message;
- then pass the history user prompts and AI assistant replies as role messages (up to the most recent 10 messages total);
- then add the current user prompt role message that needs an answer.
I coded a particular way for the forum so that future features like calling functions and their returns are easily recorded in chat history, which don’t have a distinct input/output pairing like the user messages. The history being a python dictionary, other metadata like input or reply tokens can be added to chat history items after receiving the successful AI response or using tiktoken. The “chat” dictionary is the full session history in case one wanted to save or reload it by command.
Proper API error handling and retrying is alone at least twice as many lines of code as I demonstrate.
My own local API chatbot is 1500+ lines of GUI python that someday I’ll continue plugging away at under the hood, after tedious GUI widgets like an expanding input box, option to send only after multiple return presses (or “submit” and resubmit like playground or just add to history), expanding panes, colored status bar with actions based on errors, font and element resizing, auto-collapsing editable and rearrange-able history, live greying of unpassed history adaptive to token sliders and input, and other non-features.
2 Likes
Hi,
When pass related prior exchanges, can I only send the Vector Database Links of related exchanges? Or have to send the whole related Text Contents ?
As I remember openAI doesn’t exept external links.
A vector database for conversation would naturally store the individual conversation turns. Embeddings is a special search method that looks not just for keywords, but for meaning. One could then retrieve older baseball chat from the conversation database if the subject being discussed has returned to baseball.
One could also have those longer conversation exchanges summarized or given context by another AI, so that they can be more compact or stand alone.
There is no “link”, one would provide the past conversation turns just as if they actually happened (with the appearance of other missing conversation if you were to read the new transcript the AI receives).
1 Like
Is there any model which preserve previous chat like chat gpt 4?
Hi and welcome to the Developer Forum!
The is currently no model that retains context across API calls.
