Can ChatGPT respond with citations?

[myreille]

If, for example, I’m doing research and I would like to also note my sources, can ChatGPT provide citations along with its responses?

[lmccallum]

I think the answer is “to some extent.” Here is an example:

[myreille]

Makes sense, thank you!

[SomebodySysop]

I’ve been looking into this myself. Did you ever find any solution?

Citation References in Fine-tuned Model Responses

I used to develop regulatory infobases (information databases) using Folio Views back in the day. Basically, these were huge databases of information designed to be keyword searched, returning links to the relevant sections in the documents. I’m looking to experiment with something like that using OpenAI. The differences I understand are that a) an AI system like OpenAI is not doing keyword seaches and b) It not so much retrieving excepts as interpreting context. Or, something like that. Wh…

[chesketh]

I have been studying this since the early days of the OpenAI playground. It is rare to get back non-hallucinated scientific literature citations with any model from GPT-3.5 and earlier. It will often use real titles of papers, but hallucinate the journal, authors, volume, page numbers, doi etc… The title is occasionally a hallucination as well. Generated citations often appear very real since it will mix up known authors, prominent journals (esp. NEJM), and familiar titles.

It is only with GPT-4 that I have been able to elicit actual citations that exist. I am finding that of the 6 standard elements of a citation (authors, journal, title, volume, page numbers, doi), usually 4/6 or 5/6 of the elements are correct. Usually 1-2 authors will be mixed up or completely hallucinated and the volume and page number are often incorrect. The doi, however, is often correct, which has important functional implications. Overall, huge step forward for OpenAI LLMs; however you still can’t fully trust the citation and all citations must be cleaned thoroughly.