NodeJS multiple requests until it's complete?

My first GPT-3 project is to pull some basic JSON from a body of text. Using NodeJS.

In the Playground:

The completion works great. I have to keep pressing the “Submit button” though, to get it to finish.

Take at least 5 times. I guess this is because of the token limitations on each request.

In NodeJS:

Using the same prompt that worked in Playground (which required multiple Submits), would the equivalent way in a programatic NodeJS environment be to just keep submitting requests? Or like maybe partial requests? Maybe keep stringify-ing the concat’d completion until it is a valid JSON object?

Are there any pointers, articles, resources that could point me in the right direction for this sort of thing?

Welcome to the community!

What model and settings are you using. What do you have max_tokens set to?

1 Like

I’m using text-davinci-003

const { Configuration, OpenAIApi } = require("openai");

const configuration = new Configuration({
  apiKey: process.env.OPENAI_API_KEY,
const openai = new OpenAIApi(configuration);

const response = await openai.createCompletion({
  model: "text-davinci-003",
  prompt: "Summarize the TEXT below into a JSON with exactly the following structure { FirstName, LastName, Email }: <TEXT INSERTED HERE>",
  temperature: 0,
  max_tokens: 60,
  top_p: 0,
  frequency_penalty: 0,
  presence_penalty: 0,

I’m playing with max_tokens in hopes it will return the entire completion in one request.

EDIT: Yes it works!!

I pasting in the prompt above (including a 5000 character string) in the tool.

It’s about 1,993 tokens.

So I bumped up the max_tokens to 2k.

But in cases where the input text a larger, pushing the total allowed tokens above 4k. I guess this where I’d need to do more than one request / completion? Not sure how to do that yet…

1 Like

Nice. Just remember that the 2k max tokens is for prompt+output. Max_tokens can somewhat control the output but not exactly all the time.

What you’ll likely need to do is chunk it up to 1k or 1.5k tokens and do it a bit at a time.

Good luck!

I think I see what you mean…

Split up the input text if it exceeds N characters? Then make separate requests?

If user adds 5000 characters, split it up and make 5 requests with 1000 characters.

Yeah, or just limit user to 1000 tokens until the context window is larger…