Stuck at "Create the fine-tuning datasets.."

system · December 21, 2022, 11:45am

openai/openai-cookbook/blob/838f000935d9df03e75e181cbcea2e306850794b/examples/fine-tuned_qa/olympics-3-train-qa.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<span style=\"color:orange; font-weight:bold\">Note: To answer questions based on text documents, we recommend the procedure in <a href=\"https://github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb\">Question Answering using Embeddings</a>. Some of the code below may rely on <a href=\"https://github.com/openai/openai-cookbook/tree/main/transition_guides_for_deprecated_API_endpoints\">deprecated API endpoints</a>.</span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Train a fine-tuning model specialized for Q&A\n",
    "This notebook will utilize the dataset of context, question and answer pairs to additionally create adversarial questions and context pairs, where the question was not generated on that context. In those cases the model will be prompted to answer \"No sufficient context for answering the question\". We will also train a discriminator model, which predicts whether the question can be answered based on the context or not.\n",
    "\n",
    "We will add hard adversarial examples as well, which will be based either on semantically similar sections, or neighbouring sections, originating from the same article."
   ]
  },
  {

This file has been truncated. show original

I’m stuck at “Create the fine-tuning datasets for Q&A and discriminator models.” in the olympics 3 documentation.
It uses the deprecated API endpoint, search.

Does the olympics 3 documentation need to be updated? The " To answer questions based on text documents, we recommend the procedure in Question Answering using Embeddings" is confusing me. Is it a followup or a replacement for the olympics 3 documentation?

I would like to fine tune model specialized for Q&A chat bot. I have done the olympics 1 & 2. I need technical support.

Topic		Replies	Views
Question Answering fine tuning example notebook not found Documentation	1	668	February 22, 2023
Chnages to fine-tuning documentation API	2	443	August 4, 2022
Fine Tuning a Chatbot to provide answers from a specific dataset API	6	3055	December 17, 2023
Finetune Error Stream interrupted (client disconnected). (pending status) API fine-tuning , api	15	703	August 11, 2023
Fine_tunes prepare_data No Response API	2	2285	March 9, 2023

Stuck at "Create the fine-tuning datasets.."

Related Topics