Examples are not working properly

API
1

I am new to OpenAI. I’m trying to reproduce the examples, and so far, many of them do not replicate the sample response shown in the example page.

For example: https://beta.openai.com/examples/default-tldr-summary is supposed to output something like

" A neutron star is a star that is so dense that it has collapsed into a sphere the size of a city."

but instead I am getting stuff like:

“A neutron star is a type of stellar remnant that can result from the gravitational collapse of a massive star during a Type II, Type Ib or Type Ic supernova event. Neutron stars are the smallest and densest stars known to exist. They result from the supernova explosion of a”

which is kind of re-phrasing the input, not at all a summary.

I didn’t modify the model parameters at all, and running multiple times gives somewhat different results (due to temperatore=0.3 I suppose) but most are worse than the output I mentioned, instead they tend to just echo the input text (not even re-phrasing)

This same issue is occuring with other examples, such as:

I am guessing that the model must have changed since the samples were created.

1 Like
2

If I switch the https://beta.openai.com/examples/default-tldr-summary to instruct-davinci it works better.

But the https://beta.openai.com/examples/default-ad-product-description example gives a poor output response regardless of the model chosen.

2 Likes
3

Welcome to the wonderful world of GPT-3 prompts! :wink:

Have you tried changing up the prompts at all to improve them? I’m not sure of the examples - when they were created, etc. Adjusting them a bit might be your best bet.

I just looked and the ad example only uses a one-shot (ie one example)… You might try doing two or even three. I bet it would improve the output for you.

Please let us know how it goes.

5

Hi @jasons, yes as @PaulBellow mentioned, the first thing to try would be changing the prompt itself, and then messing with the parameters.

The lower temperature you use, the more frequently you’ll get the same (ish) result.

2 Likes
6

The problem is that some of the examples just do not work. I don’t mind going through these and experimenting. I’m going through each of them and spending the time to figure them out. But some, like the “default-ad-product-description” just don’t work regardless of how I try massaging the input prompt.

I don’t think it’s unreasonable to expect an official samples to be reproducible, at least approximately. Maybe the model changed and so these examples no longer work. If they could get fixed, or removed I think that would be the best.

1 Like
7

Thanks for the feedback, we can look into it! Overall, GPT-3 is a generative engine, and a large part of the fun is trying to find ways to get it to create the output you desire :slight_smile:

2 Likes