Optimizing WebVerify Feature Performance for Faster Results

Community
1

Hi,

I’m working on a feature for WebVerify. Here’s my process:

  1. Generate search terms using OpenAI.
  2. Make a request via the Google Search API.
  3. Scrape the returned URLs.
  4. Summarize the content using ChatGPT.
  5. Send back the results.

But the whole process is really slow.

Is it because I’m using Python, or do they have a more optimized method? How do tools like Perplexity or Morphic.sh manage to be so fast?

Thanks for your feedback!