I have build an API to fetch data from openai’s knowledge in json format.
i would like to extend its knowledge primarily to a specific website(url).
How can i achieve that ?
What do you mean by this?
getting response in json mode.
I know that. I don’t understand what you mean by openai’s knowledge. Do you mean the response you get from Chat Completion API?
Also, do you want your API Hit to browse the internet as well? Is that what you want?
Yes I mean the Chat completion response.
I would like to add a specific website to openai’s knowledge.
I was asking if there is some way i can do that by passing the url to open ai or with function calling or something
No, GPT models cannot read Web Page from a given URL. The only way I could imagine would be scrap the Website using Python Libraries like BeautifulSoup4, Organize that data and then pass it to GPT model, either using Completion APIs or Assistant APIs.
1 Like
Yes right, but from URL scraping data is too large to send as a context.
You’ll need advance good scrapping techniques to filter the data. Like Fetch only content that is inside paragraph tags. If the information is still too large, create a file out of the scrapped data, upload it to OpenAI vector store and run OpenAI assistant on it.