It seems like you’re trying to use OpenAI’s GPT-3.5-turbo model with Node.js to generate responses
Here’s a quick review of your code and some suggestions on how to improve it:
Make sure that you’ve imported the necessary modules at the beginning of your script:
const { Configuration, OpenAIApi } = require('openai');
const config = require('./config'); // Replace 'config' with the actual path to your configuration file
Regarding your initOpenAIGPTConfiguration function, there are a few modifications that could be made for better readability and error handling:
Overall, your code looks good, and it should work as intended. These suggestions mainly focus on improving readability and error handling. Don’t hesitate to ask if you have any questions or need further assistance!
thanks johnwood for quick reply, after 500 I tried with text-davinci-003 and it went well, I will go over your suggestions and re run and see if anything I may trouble you again
I am trying to call { openai.createChatCompletion( )} in a loop for ~20 prompts, but soem time it goes fine sometimes it gives me 429 status code and some time 502 when I hit api via postman. Am I missing something. Below is code where sectionList is 20.
const promises = await sectionList.map(async ( elem) => {
if(model === “text-davinci-003”){
const requestData = {
temp : parseFloat(temp),
model : model,
frequency_penalty : parseFloat(freqPenalty),
presence_penalty: parseFloat(presencePenalty),
prompt : You are a senior business consultant writing a draft business plan for an SME business owner. Each business plan has multiple sections that require information. The business name is ${businessName} and the industry description is ${industryDesc}. For each section, if you have specific factual information on the business, use it along with the industry description to generate your responses. Otherwise, use only the industry description. Strive to provide comprehensive responses to all sections while minimizing repetition. Provide responses in UK English. Section: ${elem.section} Question: ${elem.question}
}
const response = await module.exports.initOpenAIConfiguration(requestData).then((result) => {
if(result){
let responseList = {}
result = result.split(‘\n’).filter(n=>n)
//return result
responseList[elem.section] = result
return responseList
}
});
return response
}
It seems that you might be encountering rate-limiting issues (status code 429) and occasional server errors (status code 502). Here’s what you can do to handle these issues:
Rate Limiting: Add a small delay between API calls, like using setTimeout or another method to add a pause. This will help you stay within the rate limits imposed by the API.
Error Handling: It’s important to handle any errors returned by the API gracefully. You could implement retry logic with exponential backoff for status codes like 429 and 502, giving the API some time before trying again
Here’s a modified version of your existing code snippet with added error handling and rate limiting:
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
const processSection = async (elem) => {
if (model === "text-davinci-003") {
const requestData = {
temp: parseFloat(temp),
model: model,
frequency_penalty: parseFloat(freqPenalty),
presence_penalty: parseFloat(presencePenalty),
prompt: // ... Your prompt ...
};
let retries = 3;
let response;
while (retries > 0) {
try {
response = await module.exports.initOpenAIConfiguration(requestData);
if (response) {
let responseList = {};
response = response.split('\n').filter(n => n);
responseList[elem.section] = response;
return responseList;
}
} catch (error) {
if (error.statusCode === 429 || error.statusCode === 502) {
await sleep(1000 * Math.pow(2, retries)); // Exponential backoff
retries--;
} else {
throw error;
}
}
}
throw new Error('API call failed after multiple retries');
}
};
(async () => {
const promises = sectionList.map(async (elem) => {
const result = await processSection(elem);
await sleep(200); // Adding a small delay between API calls
return result;
});
const responses = await Promise.all(promises);
console.log(responses);
})();
This code adds error handling and rate limiting to your existing implementation. Be sure to adjust the sleep time for backoff and between API calls as needed.
Thanks John for quick response, will try your suggestion and let you know.
To avoid rate limiting issue can we do something with account, I mean if we have paid account will that give me stability or I am missing sothing here.
Yes, upgrading to a paid account can help alleviate rate-limiting issues. Most APIs, including OpenAI’s GPT-3 API, have different rate limits based on the type of account you have (free tier vs. paid plans). Paid accounts typically have higher rate limits and offer more stability for making API calls
However, even with a paid account, it’s still important to implement proper error handling and rate limiting in your code as suggested earlier. This will ensure your application can gracefully handle unexpected errors or temporary downtimes.
Feel free to try out the suggestions provided earlier and let me know if you need any further assistance.
Hi John, so trying your options goes fine but when we try via curl, browser or postman response is not consistent I get 502 not from openAI but as we are trying to hit openAI in a loop with almost ~28 prompts with retry and sleep, client (postman, curl) responding back with 502. Can’t we sent all the 20 prompts as a single to openAI, or anything which you suggest.
Ideally what should be the response time if we use gpt-3.5-turbo vs text-davinci-003, I see every request is taking more than 30 seconds irrespective of paid or free account. Am I missing anything.
I have a paid account and spend quite a bit every month, but I’m getting 502s and 429s as well, and I’m nowhere near the rate limit. This is including exponential backoff. I haven’t gotten a successful gpt-4 response in almost 24 hours and about 1 out of every gpt-3.5-turbo requests fail.