After much fiddling, I still haven’t had success with negative examples. I found this example on github which mentions negative examples, but not suitable for the use case described above:
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<span style=\"color:orange; font-weight:bold\">Note: To answer questions based on text documents, we recommend the procedure in <a href=\"https://github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_embeddings.ipynb\">Question Answering using Embeddings</a>. Some of the code below may rely on <a href=\"https://github.com/openai/openai-cookbook/tree/main/transition_guides_for_deprecated_API_endpoints\">deprecated API endpoints</a>.</span>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 3. Train a fine-tuning model specialized for Q&A\n",
"This notebook will utilize the dataset of context, question and answer pairs to additionally create adversarial questions and context pairs, where the question was not generated on that context. In those cases the model will be prompted to answer \"No sufficient context for answering the question\". We will also train a discriminator model, which predicts whether the question can be answered based on the context or not.\n",
"\n",
"We will add hard adversarial examples as well, which will be based either on semantically similar sections, or neighbouring sections, originating from the same article."
]
},
{
show original
1 Like