Hello, i have already developed and have the GPT api working, but as it is my very first time dealing with AI related things, i would like to know if there has something to improve in my setup.
AI Setup
I’m running a GPT assistant to provide support for customers that ask questions about my resources.
I created one assistant and in the “Instructions” i put a very long text explaining every aspect of supporting, also i made it clear to not answer questions that are not related to resources. I gave some overview on how to proceed with some kind of situations.
I have enabled the “File Search” where i’ve uploaded the documentation of all the resources (as .md), and i’ve also uploaded the configuration files of them (as .txt). (it has 2MB now). Like this:
Issues i’m having:
With these settings, the AI provide some good answers but also sometimes he hallucinates.
Issue 1
For example, i have several resources, if a customer asks about a specific one, sometimes he gets info about another resource and answer based on that other.
Issue 2
Another issue, not sure if its related but i’ve put this in the Assistants “Instructions” text:
2. File Storage: Always use the files in file storage to find answers to the user’s questions. Specially for the common issues.
3. Script Identification: Always start by asking the user for the name of the script they are inquiring about. Only provide support after the user clearly specify what is the script or question.
Then the AI gets stuck when it says it’ll search in the uploaded files, i believe the API should return something else but i’m not catching. I noticed that usually this issue happens after he asks the name of the script (i had to send another prompt “then?” so it could return me an answer). See below the conversation:
My js API code
// Pass in the user question into the existing thread
try {
await openai.beta.threads.messages.create(
threadId,
{ role: "user", content: content },
);
} catch (error) {
clearInterval(sendTypingInterval);
console.error("OpenAI Message Create Error:", error);
if (Utils.channelExists(channel))
message.reply("There was an error sending your question. Please try again later.");
return;
}
// Use runs to wait for the assistant response and then retrieve it
let run;
try {
run = await openai.beta.threads.runs.create(
threadId,
{
assistant_id: "asst_xxxxxx",
model: gptModel,
},
);
} catch (error) {
clearInterval(sendTypingInterval);
console.error("OpenAI Run Create Error:", error);
if (Utils.channelExists(channel))
message.reply("There was an error initiating the response generation. Please try again later.");
return;
}
// Polling mechanism to see if runStatus is completed
const pollingInterval = 2000;
const maxPollingTime = 60000; // 1 minute
let elapsedTime = 0;
let runStatus;
do {
if (elapsedTime >= maxPollingTime) {
clearInterval(sendTypingInterval);
if (Utils.channelExists(channel))
message.reply("The assistant took too long to respond. Please try again later.");
return;
}
await new Promise(resolve => setTimeout(resolve, pollingInterval));
elapsedTime += pollingInterval;
try {
runStatus = await openai.beta.threads.runs.retrieve(threadId, run.id);
} catch (error) {
clearInterval(sendTypingInterval);
console.error("OpenAI Run Retrieve Error:", error);
if (Utils.channelExists(channel))
message.reply("There was an error checking the response status. Please try again later.");
return;
}
} while (runStatus.status !== "completed");
// Get the last assistant message from the messages array
let messages;
try {
messages = await openai.beta.threads.messages.list(threadId);
} catch (error) {
clearInterval(sendTypingInterval);
console.error("OpenAI Messages List Error:", error);
if (Utils.channelExists(channel))
message.reply("There was an error retrieving the response. Please try again later.");
return;
}
// Find the last message for the current run
const lastMessageForRun = messages.data
.filter(
(message) => message.run_id === run.id && message.role === "assistant",
).pop();
clearInterval(sendTypingInterval);
if (!lastMessageForRun) {
if (Utils.channelExists(channel))
message.reply("The chat is having some troubles at the moment. Please try again later.");
return;
}
// Split message into chunks because discord has character limit per message
console.log(JSON.stringify(lastMessageForRun.content, null, 2));
for (const messageForRun of lastMessageForRun.content) {
const responseMessage = messageForRun.text.value;
const chunkSizeLimit = 1900;
for (let i = 0; i < responseMessage.length; i += chunkSizeLimit) {
const chunk = responseMessage.substring(i, i + chunkSizeLimit);
if (Utils.channelExists(channel))
await message.reply(chunk);
}
const logText =
`
- **User:** <@${message.author.id}> ${message.author.id}
- **Question:** ${message.content}
- **Answer:** ${lastMessageForRun.content[0].text.value}
- **Attachments:** ${attachments.length > 0 ? attachments.join(", ") : "None"}
`;
sendTextLogToChannel(process.env.CANAL_LOGS_OPENAI, logText, client);
}
). What I would do is for each chapter in the Mac document I would insert in each section