That tutorial is dated, and the first section that had to do with language models will get you right to code that doesn’t work…
Traceback (most recent call last):
File "<pyshell#157>", line 1, in <module>
from openai.embeddings_utils import distances_from_embeddings
ModuleNotFoundError: No module named 'openai.embeddings_utils'
So you would have to build using openai<=0.28.1 if you wanted to copy paste.
Instead of those utilities, The AI can help you store embeddings returns in ndarrays and write a numpy dot product tensor comparison though, the line of which is shorter than this very line.
However the whole premise launched at the start is misguided. You don’t need a web site scraper if you are the proprietor of the knowledge within, in text form. You can go right to your database or source text to make information the AI can easily consume. Then a change management system so embeddings and text are kept up to date.
I would use the tutorial like the obscure language with no application that you were made to program with to learn computer science fundamentals. That the tutorial is metacode that needs to be filtered through the lens of your data you have direct access to and the newest python methods going forward.
The idea of the website is for a service that we want to implement in the company, that the client inserts the website and through a chat they can ask questions and respond by taking information from the website.