Hi! I’m implementing the web_seach tool in Response API. Could it be used for web scraping? I’ve seen that it returns data like news and internet data, but it doesn’t require querying a product listing, for example. Does anyone use it in that way? Thanks!
2 Likes
Hello, Celia.
I developed a tool using Openchai and Mistral 7B to search for job vacancies on websites across the internet.
Roll your own. I use jina.ai … they have an API for crawling that usefully returns markdown (I’m not affiliated). There are a few others too (e.g. Firecrawl). Support good third parties!
2 Likes