Citation feature for document based answering

API
,
1

Hi there,

for all of you who’re using the chat completions endpoint - has anyone managed to build an RAG with citations? I have a basic, naive RAG setup using embeddings but wondering how one might implement the citation feature. The assistants api provides it by default and perplexity has it as well.

I wonder if its just prompt engineering under the hood.

2

It’s a combination of a few things:

  • prompt engineering is key, yes
  • you also need to structure the output in a way that you can parse out citations
  • parsing that structured output gets tricky

Here’s some example code and what it looks like when deployed