API gives me too similar of results when I batch

I ask something like: “Write 3 points on why WEBSITE is great, and ask a question a support questions.”


If I request 1 at a time with short gaps, they are pretty different. But if I request large batches of 500 or 1000, they start to become very similar. Using the same initial lines, of exactly the same phrases. 30 to 50% ends up the same, with what I would call mostly filler words like “I would ask WEBSITE X.” or “The first point would be” or “And Finally the last point is”

Is there any way to avoid this?

Sorry, I mis-read your post. There shouldn’t be any sort of difference based on the timing. Am I correct in believing that you are requesting 500 - 1000 batches using 500 - 1000 different websites? How are you batching each request?

Are you aware that GPT cannot actually view the website? You would be better off including a blurb of the website’s about me, or something similar. Or are you using a plugin to view the website?

Welcome to the community.

Have you considered the temperature, frequency_penalty and presence_penalty. Each by itself is likely to have an impact on the variability of the response.

Good luck :four_leaf_clover:

What model are you using? What does the code calling the endpoint look like? Can you share a couple of response objects?