Strange behavior of a fine tuned model

Mikiane · December 31, 2022, 4:42pm

Hi, all

I have fine tuned davinci with a set of data (prompt: question ? / completion: answer) and I get a strange result.

When I use the fine tuned model using some prompts that exist in the dataset, I get a completely different result than the completion it is trained for.

Do you have an explication? (Temperature = 0 / Top P = 1)

Mikiane · January 2, 2023, 1:04am

I retried with another dataset today.

I have fined tuned davinci with a dataset of +400 lines (~500 tokens / line) extracted from one of my books and the result is a mess.

I am actually chating with a fool…

anybody here is experimenting this ?

I am trying to build an expert Chatbot on the topics I wrote in my books…

Thanks for your help

Ginger · January 2, 2023, 6:17am

Im following thisn topic since I may be interested in the reason of why this behaviour is given.

Mikiane · January 2, 2023, 10:43am

After some tests… Things are getting a little bit better…
Here are some lessons (for now):
1/ Fine-tuning a model doesn’t prevent prompt design. The prompt has to be as efficient as possible, even with a fine-tuned model.
2/ Using low temperature, high Top P, High Frequency, and presence Penality seem a good option
3/ defining a stop sequence is also a good choice. For classical contents (extracts from books) I use a double “return”.

WIP…

alan2here · January 2, 2023, 5:26pm

There are some subjects like Math (arguing with it about graph theory, no numbers or equations, but it doesn’t really understand the concepts very well) where it really fails.

I’m not certain, but I think fine-tuning whittles it down rather than teaches it new things, so if it doesn’t know something I don’t think you can teach it to it with fine-tuning.

Maybe you could get your books into the next round of GPT-3 training? I don’t know how much of Google Books repository is included in the AI’s training data.

kevin6 · January 2, 2023, 7:20pm

If you would like to extract information from a book, consider embedding models rather than fine-tuning a model.

You may also find the following link useful:

github.com

openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c4ca8276-e829-4cff-8905-47534e4b4d4e",
   "metadata": {},
   "source": [
    "# Question Answering using Embeddings\n",
    "\n",
    "Many use cases require GPT-3 to respond to user questions with insightful answers. For example, a customer support chatbot may need to provide answers to common questions. The GPT models have picked up a lot of general knowledge in training, but we often need to ingest and use a large library of more specific information.\n",
    "\n",
    "In this notebook we will demonstrate a method for enabling GPT-3 able to answer questions using a library of text as a reference, by using document embeddings and retrieval. We'll be using a dataset of Wikipedia articles about the 2020 Summer Olympic Games. Please see [this notebook](fine-tuned_qa/olympics-1-collect-data.ipynb) to follow the data gathering process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "9e3839a6-9146-4f60-b74b-19abbc24278d",
   "metadata": {},
   "outputs": [],

This file has been truncated. show original

Topic		Replies	Views
Struggling with poor performance on fine-tuned davinci model API	15	2572	December 20, 2023
Fine Tuned Chatbot forgets how to output summary of conversation API	9	1738	December 18, 2023
First Fine Tune was kind of disappointing? API	1	510	February 6, 2024
Got awful results after fine-tuning API	11	3101	December 1, 2022
Building the first fine-tuned model API	5	963	December 27, 2023

Strange behavior of a fine tuned model

Related Topics