Best approach for factual/verfiable data?

What are some good prompt approaches for factual/verifiable text generation? (like: historical, medical, tech, research, and even data from wikipedia)

I’m having a hard time wrapping my head around this problem and I’d appreciate any insightful action-oriented input!

2 Likes

Well, this is one reason why OpenAI requires human input into the generation workflow. The model can generate too much fake info.

You only control the prompt and the settings, so try to play with those. For example, lower temperature and strict enough prompt would help to get more consistent and predictable results.

I’m not an ML engineer, but I can imagine one can solve this issue by comparing the output against a set of facts from a facts-based knowledge base (knowledge graph?) and classify it as true or not.

  • Get the output
  • Extract the entities and connections between them
  • Find the claims the text makes (based on connections between the entities)
  • Compare the information with your knowledge base
  • Decide if the text is trustworthy enough

Not a task for a regular app developer though :smiley:

2 Likes

Hi @cristi! Please also take a look at our suggestions for Factual Responses.

4 Likes

thanks I already know that…

thanks kearonis! some of your suggestions might be helpful! :smiley:

@cristi i access third party apis with queries, then use the content from those responses in the GPT3 prompt.

WolframAlpha.com - a system and API that provides computer algebra computation, 1000s of algorithmic answers to queries, verified data in a variety of domains and easy to use interpreted output data.
Wolfram|Alpha APIs: Computational Knowledge Integration
Wolfram|Alpha Full Results API Explore

WikiData - a system and API that provides data look ups from Wikipedia. The data isn’t always verified, but in some
Wikidata API
https://query.wikidata.org

Wikipedia API -
https://www.mediawiki.org/wiki/API:Tutorial

PubChem - US Government resource for chemicals
https://pubchem.ncbi.nlm.nih.gov/ and PUG REST

Google Knowledge Graph

TextRazor
https://www.textrazor.com/

DoWhy from Microsoft
https://microsoft.github.io/dowhy/

6 Likes