Web scraping in Responses API

Hi! I’m implementing the web_seach tool in Response API. Could it be used for web scraping? I’ve seen that it returns data like news and internet data, but it doesn’t require querying a product listing, for example. Does anyone use it in that way? Thanks! :smile:

2 Likes

Hello, Celia.
I developed a tool using Openchai and Mistral 7B to search for job vacancies on websites across the internet.

Roll your own. I use jina.ai … they have an API for crawling that usefully returns markdown (I’m not affiliated). There are a few others too (e.g. Firecrawl). Support good third parties!

2 Likes