# Fine-tuning on conversations **URL:** \ **Category:** API\ **Created:** [January 13, 2023, 2:54pm UTC](https://community.openai.com/t/fine-tuning-on-conversations/35471 "2023-01-13T14:54:11Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Helveticus](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/helveticus/32/17122_2.png) [@Helveticus](https://community.openai.com/u/Helveticus)\ **Post date:** [January 13, 2023, 2:54pm UTC](https://community.openai.com/t/fine-tuning-on-conversations/35471/1 "2023-01-13T14:54:11Z")
Hello I have a dataset consisting of dialogues between two people which I would like to use for fine-tuning GPT. Please see below for two example dialogues. The dialogues vary in length and can be longer than the examples. Is the format of the conversations ok? For fine-tuning, should I just concatenate all conversations or do I have to use a separater between the conversations (if yes, which separater)? First Dialogue: > user1: > Hey there. What’s up? > > user2: > Not much, just hanging out. What about you? > > user1: > Just thinking about what I’m going to do this weekend. You? > > user2: > Probably just relaxing. What do you have planned? > > user1: > I’m thinking about going to the beach. It’s supposed to be nice this weekend. > > user2: > That sounds like a great plan! Have you been to the beach recently? > > user1: > Not in a while. It would be nice to get out and enjoy the sun. > > user2: > Definitely! I’m sure it’ll be a great time. Do you have any other ideas for the weekend? Second Dialgoue: > user1: > Good morning. What is your profession? > > user2: > Good morning. I’m an accountant. What about you? > > user1: > I’m a software engineer. How long have you been an accountant? > > user2: > I’ve been an accountant for about five years now. What about you? How long have you been a software engineer? > > user1: > I’ve been a software engineer for three years. What do you like most about accounting? > > user2: > I like how challenging it can be. There’s always something to learn or something new to figure out. What do you like most about software engineering? > > user1: > I like how creative it can be. I get to come up with new ideas and new ways of solving problems. It’s a great feeling when you can come up with something that works. ---
**Author:** ![curt.kennedy](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/curt.kennedy/32/709249_2.png) [@curt.kennedy](https://community.openai.com/u/curt.kennedy)\ **Post date:** [January 13, 2023, 3:01pm UTC](https://community.openai.com/t/fine-tuning-on-conversations/35471/2 "2023-01-13T15:01:40Z")
For a generic back and forth, try creating a jsonl file like this: {“prompt”: “Hey there. What’s up?\n\n###\n\n”, “completion”: " Not much, just hanging out. What about you?“} {“prompt”: “Not much, just hanging out. What about you?\n\n###\n\n”, “completion”: " Just thinking about what I’m going to do this weekend. You?”} Note how they have 50 percent overlap. I’d be curious to see what you come up with! ---
**Author:** ![Helveticus](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/helveticus/32/17122_2.png) [@Helveticus](https://community.openai.com/u/Helveticus)\ **Post date:** [January 14, 2023, 11:12am UTC](https://community.openai.com/t/fine-tuning-on-conversations/35471/3 "2023-01-14T11:12:20Z")
Thank you very much. Could it be a problem that with your version we only have one previous message taken into account but usually for a conversation many more previous message are important for the context? ---
**Author:** ![curt.kennedy](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/curt.kennedy/32/709249_2.png) [@curt.kennedy](https://community.openai.com/u/curt.kennedy)\ **Post date:** [January 14, 2023, 2:52pm UTC](https://community.openai.com/t/fine-tuning-on-conversations/35471/4 "2023-01-14T14:52:39Z")
You are correct, there is only the previous input taken into context to produce the next output. But your example seemed to have a gentle back-and-forth feel to it, without a deeper prior context. To have a deeper context ‘virtual you’, I think you need at least two fine tuned models. And the more I think about it, I think you also need several classifiers running in the background, as well as several non-AI based RegEx pattern cross-correlation engines. Here is at least my early thoughts on creating a deeper ‘virtual you’ without the extra stuff I just mentioned above: > [@Extracting Personalities from past Conversations?](https://community.openai.com/t/extracting-personalities-from-past-conversations/33551): > > I am wondering what others ideas are on extracting ‘personalities’ using past conversations as training data. For example, suppose we are only interested in one persons personality based in N past conversation transcripts they have had. Here is a simple example: Let’s say I want to replicate a bot that responds like Dieter (Mike Meyers) from the SNL skit “Sprockets!”. There are 14 “Sprockets” skits from 1989 to 1997. So not a lot of data, although there is also a movie screenplay out there … ---
**Author:** ![micamecava](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/micamecava/32/63892_2.png) [@micamecava](https://community.openai.com/u/micamecava)\ **Post date:** [April 3, 2023, 8:50pm UTC](https://community.openai.com/t/fine-tuning-on-conversations/35471/5 "2023-04-03T20:50:22Z")
Did you figure anything out about this @Helveticus ? @curt.kennedy ? I’m also looking for fine-tuning gpt3 on the conversational data where the data is highly contextual. ---
**Author:** ![curt.kennedy](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/curt.kennedy/32/709249_2.png) [@curt.kennedy](https://community.openai.com/u/curt.kennedy)\ **Post date:** [April 4, 2023, 2:34pm UTC](https://community.openai.com/t/fine-tuning-on-conversations/35471/6 "2023-04-04T14:34:38Z")
@micamecava I found a possible personally extracting solution over here [Extracting Personalities from past Conversations? - #6 by curt.kennedy](https://community.openai.com/t/extracting-personalities-from-past-conversations/33551/6) ---
**Author:** ![EricGT](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/ericgt/32/20571_2.png) [@EricGT](https://community.openai.com/u/EricGT)\ **Post date:** [December 14, 2023, 11:07pm UTC](https://community.openai.com/t/fine-tuning-on-conversations/35471/7 "2023-12-14T23:07:38Z")