Is it possible to generate content based on a website?

API
1

Is there a way to generate content based on a website? Asking the ai something like “create a summary of the website xyz” dies not work.

2 Likes
2

If you wanted to do this for a single website, you’re probably best off copying and pasting the website text into GPT-3.

If you wanted to automate the process, you would code an application to scrape the desired website, send the website’s text to GPT-3 via the API, and then ask GPT-3 to generate content based off the scraped website text.

2 Likes
3

The language models (like ChatGPT) don’t actually have internet access although they sometimes say they do.

I agree that your best bet would be to slurp up all the content on a webpage and summarize that way.

What are you trying to build?

2 Likes
4

I’m working on a similar project at the moment.

Are you familiar with BeautifulSoup, a Python library for pulling data out of HTML?

High level steps that I’m taking:
Scrape the data, append ‘Tl;dr’ to the end, and feed that prompt into the OpenAI API completion request.

Is this helpful?

2 Likes
5

@PaulBelow I am trying to build a question generation tool which can use a website as source - in flutter.
@light.iron Yes… found something that will work for me.
https://www.geeksforgeeks.org/web-scraping-in-flutter/ will be the way I am going to test.

1 Like