Using /answers to query a hierarchial knowledgebase

Hey there,

I was trying to get GPT-3 to answer some questions from a knowledgebase using the /answers endpoint. The data is ordered hierarchically, so I fed it as markdown, where every main header is its own line in the JSONL file I upload. I provided an example context, which was a piece of markdown, and 3 example questions with answers. However, GPT-3 gives me incomplete or untrue answers. The structure of the markdown is as follows:

# When will my package arrive?
## What time does the delivery person come exactly?
We cannot tell you when your package will arrive...
## Subtopic / question
* foo
* bar
* baz

And when I ask: “When whill my package arrive?” it tells me: “The delivery person will be there at 10:00”.

So my question would be if there is a better way to represent hierarchical data for GPT-3, or are there specific examples which would show him what he should do?

2 Likes

Thank you @m-a.schenk! When someone asks when a package will arrive, for now I want it to answer that they cannot tell exactly when it should arrive. A lot of the questions could be answered if the right context would be available. I followed your advice and looked at the prompt it generated and it doesn’t seem to add the right context in the prompt. It seems to match the right document, it also seems to select the right subheader, but it only puts the subheader as context, so:

Context: ## What time does the delivery person come exactly? ## What time does the delivery person come exactly? ## What time does the delivery person come exactly? ## What time does the delivery person come exactly?

However, the context should’ve been:

Context:# When will my package arrive?
## What time does the delivery person come exactly?
We cannot tell you when your package will arrive...
## Subtopic / question
* foo
* bar
* baz

How can I influence what the context should be? Maybe I formatted my markdown file the wrong way, I replaced \n at the end of each line with \\n, so I might look into that.

2 Likes

Unfortunately, I haven’t had much luck with the answers API either lol I think it’s a feedback item for them, although I tried it months ago when it first came out

So maybe instead of using the answers endpoint, I could use a search + completion call. First I search for the text which matches the question, then I use a fine-tuned model to extract text the right way from markdown.

1 Like