Fine Tuned Model Making It's Own Prompts

My fine tuned model is having a conversation with itself in the completion, and I’m trying to understand why.

I’ve trained a Davinci based model on a book of interview question/response pairs. Each question/response pair is a line in the JSON lines file with question as the prompt, and response as the completion. I used the prompt suffix “\n\n###\n\n” and the completion suffix “\n#####\n”.

When I prompted this model with a interview question after training, I was surprised to find that it not only responded with an answer to my question, but proceeded to ask itself another question, and respond to itself, and ask another question, and so on until it ran out of tokens, separating question/response with “#####”.

Here’s an example

who are you?


Nobody in particular. Just a witness. ##### what are you witnessing? ##### Everything, from nothing to nothing. ##### and where are you while witnessing? ##### I am nowhere, I am all. As I see the world, so it is. With a turn of mind, it is otherwise. I am in the world, but not of it. ##### What is the ...

I’ve not seen this in untrained models. Why does the fine tuned model prompt itself?
I read this in the Api Docs

The weight to use for loss on the prompt tokens. This controls how much the model tries to learn to generate the prompt...

Does this mean the model is trying to learn the prompt as well? Why?

Thanks for the help! Let me know if I can give you any other context.

Welcome to the forum.

Are you using this as a stop sequence when you call the API?

I am not sure of your prompt and completion format in the JSONL file. Maybe you could post a small snippet

Generally the following format works well:

prompt : “Human: Whatever your question is\nAI:”
completion : " Whatever your answer is"

Note that the two parts are tagged with Human and AI captions. The completion starts with a space. The AI: tag is at the end of the prompt. It has a \n before it.

Then when you ask the trained model a question, your prompt should be something similar to the following

“Human: This is my question\nAI:”

Your format doesn’t need to be exactly the same. But the idea is you want to teach the AI a pattern to follow. Your pattern is a human question, and waiting for a response from the ai.

If you have enough examples, the AI will use the weight of the knowledge you have given it in this format to come up with something that resembles (but maybe not exactly matches) one of your training completions.

1 Like

Thank you! I wasn’t aware of these, but it looks very promising. I’ll post back after I’ve tried it out.

Here’s a sample train entry

{“prompt”:“And not what I am?\n\n###\n\n”,“completion”:" What you are, you already are. By knowing what you are not, you are free of it and remain in your own natural state. It all happens quite spontaneously and effortlessly.\n#####\n"}

I’m still a little surprised it managed to learn the prompt as well as the completion, but I think the stop sequence should work to keep it from prompting itself.


Well done @kaplannp

You have created JSONL entries which actually follow the OpenAI guidelines.

Most folks here do not follow the guidelines.

Well done!

Note: You can experiment with different separators and stops to get different results.

And of course do not forget to use your separator when you prompt your fine-tuned model and add your stop to the completion stop parameter.


1 Like

Get rid of the \n####\n from the completion part. And add [DONE] with all caps and square brackets

The end tag is sent (silently) when the ai is streaming replies. It should help it stop

edit: changed [END] to [DONE] (after checking the API docs)

1 Like

Omega Force!

I don’t think this is always a good idea and I do not do this.

When you remove the stop and assume the [END] end tag is silently sent, you are hiding a parameter which could change in the future or perhaps is not known to another developer working on the code in the future.

It’s better, in my view, to explicitly include your stop parameter in your JSONL file and completion call, just like the fine-tuning guidelines mention.

So, in my experience in four decades of coding, it’s better to be explicit when coding so you are not scratching your head later when you forget your code / formatting /data depended on something not fully documented in the Preparing your dataset section of the docs.

I made a mistake in my previous post. Instead of [END], I meant to say [DONE]

This is not in response to your comment above - but I have edited my original post

This is mentioned under streaming in the API

I’ll leave it to you guys to take it from here

Thanks for fixing that.

Isn’t the [DONE] stop only available (according to the docs) when the stream param is set true ?

You are correct that it is used for streaming

I have seen a lot of examples where the word END in capitals has been put at the end of completions in fine-tuning files.

Then when the API is called, the stop setting is set to ‘\n’ and ‘END’

I think it is also correct that the [END] stop is only added automatically when the stream param is set true.

Thanks all! In this case, I wasn’t using streaming, but adding the stop parameter fixed things for me.