I’m looking at the answers endpoint to try to return a “factual” statement from GPT-3. Before I start uploading files, though, I thought I’d try to generate a simple example. For some reason, ada and curie and davinci get it “wrong”, while babbage nails it.
Example python request:
answer = openai.Answer.create(
search_model="ada",
model="ada",
documents=["Brian wears shorts on Thursdays.", "Brian wears pants on Fridays.", "Brian wears a kilt Monday, Tuesday and Wednesday.", "Brian like dishwashers."],
question="Does Brian wear kilts on Fridays?",
examples_context="In 2017, U.S. life expectancy was 78.6 years.",
examples=[["What is human life expectancy in the United States?", "Seventy-eight long and luxurious years."]],
max_rerank=10,
max_tokens=30,
temperature=0.0,
stop=["END", "\n", "."],
return_prompt=True
)
Answers yes.
Originally, I had “Brian wears a kilt everyday, except Thursdays and Fridays”, but even the above rewording is still not operational. Except model=babbage, which always works, irrespective of the search_model value.
finally, asking
question="It is Friday, what is Brian wearing?",
Tells me “kilt”.
Is there something off with these simple documents and sentences?