Implementing WebGPT

My use-case requires what WebGPT already does. But i did not find any API Endpoint or a way to readily use it. So i assume, this isn’t available yet.

Before i attempt to do this myself, i wanted to see if anyone has had any experience or advice on implementing feature like WebGPT.

This is exactly what i need (and my rough plan):
Sample Use Case: Create a unique article, using existing online articles as reference.

  1. A topic/subject is provided alongside with a list of URLs that contain relevant information about that topic by the user.

E.g. “Is the Keto Diet Safe”, URL1, URL2,… The user needs to research and provide the correct URLs (or this can later be integrated with Bing API)

  1. This topic/subject is first run through a (fine-tuned) prompt that outputs context information like Topic, Sub-topic, Style, Tone, Key-phrase etc.

E.g. Topic: Health, Sub-Topic: Weight Loss, Supplements, Type: Blog, Tone: Relaxed, Key-Phrase: Keto Diet

  1. The WebApp/Node.JS Code pre-processes and fetches information from the URls provided (Methods are explained here [2112.09332] WebGPT: Browser-assisted question-answering with human feedback) Have already implemented this part roughly.

  2. The extracted information (step 3) and the context information (step 2) is then used in a prompt-chain to achieve the desired results.

A better use-case for this would be to produce news articles as GPT 3 is not aware of anything after its birth. Other examples are Market Summary, Sports News, Weather News etc.


You can accomplish this by using a combination of web scraping, and embeddings

1 Like

I’ve been using a combination of GPT-3 for assessing intent, and SerpApi for performing a search (when current, post-model training, info seems desired). See Talk w/GPT-3

1 Like