Gpt-3.5-turbo and frequency/presence penalty

API
1

I need to teach my students about frequency_penalty and presence_penalty as part of a chatbot we are building using ChatGPT API’s gpt-3.5-turbo model.

The problem I am having is that when setting frequency_penalty and/or presence_penalty anywhere from -2 to 2 I am not really seeing any tangible difference in the completions.

What I would like to do is generate some text with a low frequency/presence penalty and then use the same message prompt(s) with the frequency/presence set much higher and be able to say “Hey, see what we did?” and for it to be clear. But it’s not happening?

Has anyone got any tips on how I might do this?

So far I have tried with random chat, describing cities, asking about famous artists, creating wish lists and shopping list ideas…

Or is it just that frequency_penalty and presence_penalty don’t make much odds with chatGPT (in which case I will use a davinci model for the demo instead)

Many thanks for any ideas/help!

2 Likes
2

How long is your output?

With short (under 400 or so tokens??) it’s not likely that frequency or presence penalty will be noticeable… It’s when you have longer output that you start to see it in play…

All that said, I’ve not tested those parameters a lot with gpt-3.5-turbo … yet.

Regardless, I hope this helps.

3 Likes
3

Ah interesting! Some of the output would have been more than 400 tokens, but probably not by much - which could explain the subtlety. I will experiment with longer outputs.
Many thanks!

2 Likes
4

I also didn’t see a noticeable difference when using top_p, which made me wonder if ChatGPT was in some way moving beyond needing tweaking!

5

I haven’t played with these values a ton but in addition to @PaulBellow suggestion for more tokens I would suggest giving the model a task like generate 50 ways to say "you can't buy that because you're broke" What you should notice is that for every run GPT (davinci-003 at least) seems to get stuck into a pattern where it takes the same base sentence form and just starts swapping out synonyms. Every run is a different base sentence form but it always does the same thing. My hunch is that with enough tokens, you would notice that adjusting these values starts noticeably altering those patterns… Just a hunch…

3 Likes
6

Thanks Stevenic, that actually works really well! In fact, 20 ways is sufficient. At 1.5 (freq and pres) you see a really clear difference compared to default - It starts of OK but quickly struggles to find coherent sentences and ends up actually making typos.
Here are the last 3 of its efforts:

18.This isnt worth going broke …arent many more affordable practical approaches than buying piecies past comfortability
19.Dectivefy areas where possible unneccessary splurge fall within transactions…
20.Your current fincidal state does not permit either temporary needs nor long term goals

:laughing:
Much appreciate your help and idea!

3 Likes
7

@PaulBellow Any recommendation for how to do this with the shorter completions?

8

Sorry, what was the question? Feel free to start a new thread, and I’ll try to help if I can. Busy in the workshop lately, but I still pop in to help when I can.

9

It is a bit of an older thread, especially at the pace of AI! Here is your previous reply:

From my review of the thread, my question was related to a recommendation on how to get less repetition in the cases where you are specifically asking ChatGPT to generate shorter completions. In my case, I’m targeting less than 16 tokens.

10

Ah, i see. I’d need more details and an example of your prompt, etc.

Good luck.

11

Here’s an example of one of our prompts–let me know how I can provide more detail:

{
    "model": "gpt-3.5-turbo",
    "messages": [
        {
            "role": "system",
            "content": "You are a loyal friend. You are sending a message that's less than 10 words."
        },
        {
            "role": "user",
            "content": "You: How did your meeting go today?"
        },
        {
            "role": "user",
            "content": "Friend: Quite well! Looks like we're going to move forward with the deal!"
        },
        {
            "role": "assistant",
            "content": "You:"
        }
    ],
    "temperature": 0.5,
    "max_tokens": 16,
    "presence_penalty": 0.0,
    "frequency_penalty": 0.5
}
12

I’d turn freq_penalty down to like .08 or something even lower maybe and raise temperature to .8 if possible…

What’s some example outputs where it’s too similar?

Are you sending the exact same prompt every time?

1 Like
13

Appreciate the insight!

Yes, there are often occasions where the prompts are the same or similar—and I’m looking for more varied output. An example output might be the following:

“Can we meet up today?”

14

Another thing to try is a different system message

Change that up and randomly pick a system message on the backend before you send.

ie…

$systemmessage1 = "You are a loyal friend. You are sending a message that's less than 10 words.";
$systemmessage2 = “You are an old friend who is short on time. Your messages are under 10 words.”;

etc etc…

1 Like
15

I like this idea and will give it a spin! Thank you!

1 Like