OpenAI Prototype for Website Q&A with Embeddings does not work

Community
,
1

The prototype is here: OpenAI Platform

After working through various failures and circumventing other problems, I am able to get all the code cells to run. However, However, the assistant fails to answer the questions. Either there is no response or it simply responds with “I don’t know” to everything.

Here are challenges I’ve encountered and workarounds I tried. Nothing works.

  1. The OpenAI website is too big for a crawl and the prototype eventually fails out or becomes unresponsive while doing the crawl. I tried to limit it to 1000 pages crawl. With that I’m able to get all the code cells to run. But it won’t answer the question.

  2. I’ve tried crawling much smaller websites. Still no answer to the questions.

  3. Tried different questions that were easily answered in some of the webpages crawled. For e.g. “What does do” where the named person has a web page that was crawled.

Has anybody gotten this to work? Is this prototype supported by OpenAI?

2

Welcome to the forum.

It would be easier for us to help if we could see your code.

It sounds like your embeddings aren’t being used…

3

I followed the instruction on OpenAI Platform, and copy-pasted all of the blocks verbatim from there into a single python file. You can see all of the code on that page (Expand the section for Web Crawler for that code). Hope that is helpful.

4

Sounds like you had to change the code a bit?

Do you have a concrete example?

5

  1. The Github code is at the openai-cookbook/tree/main/apps/ as web-crawl-q-and-a
    I downloaded this, am running it right now and its doing the crawl. Will monitor if it runs through fine.

  2. To answer your question, the changes i made were straightforward:
    (1) change the crawled site from “openai.com” to another site. I could see that it was crawling it fine - no errors.
    (2) limiting the openai crawl to 1000 links as follows:

Defined a variable before the while loop in the crawl(url) function:
i=1
# While the queue is not empty, continue crawling
while queue:
…
Added the following inside the while loop, at the end of the loop:
print(i)
if i == 1000:
break
#print(i)
i += 1
…

To add new questions for assistant, i just edited the questions at the end of the code file.

6

I don’t have a lot of embedding experience myself, and I’m out of ideas.

Good luck in figuring it out.

7

Q&A using Embeddings has the typical flow:

  1. prepare data for embedding. in your case, this is the result from the web crawler. check how the info are arranged, if it is logical and even you can manually understand it. Not all webpages are written properly. That is why even Bing or Bard cannot get just any website summary sometimes.

  2. send data to embeddings api and it will return vector data. save for later.

  3. receive user question, get its embeddings, process it together with the vector data from previous step. here, you should already know if you have actual hit or not, if you get any proper result.

  4. send user question, step 3 result to chat api for summary. there is an off chance that the AI will disregard the result from step 3 if it determines that the data you give does not match what was being asked.

based on what you have written, i am guessing that the result of web crawling is not good. can you verify?

1 Like