Is the OpenAI Embedding working well in the NodeJS?

API
1

Hello everyone.
I am a NodeJS developer and I love OpenAI.
But I have some problems on the OpenAI Embeddings.
I am going to implement PDF reader chatbot but when I embed the pdf data using OpenAI Embedding API and store it to the pinecone, it’s difficult to find the similar data.
Of course working well but I can’t find the result I want to look.
Is NodeJS good for that? I think I can get the better result if I use Python framework but anyway, I don’t think I am getting the best result now.
Please give me some tips if someone knows the solution.
I want to collaborate to make the wonderful world.
Thanks.

2

It’s great to hear that you’re enjoying using OpenAI! Regarding your issue with Embeddings, it’s important to note that the quality of the results depends on various factors, including the data you’re embedding.

If you feel that you’re not getting the desired results, make sure that the PDF data you’re embedding is properly preprocessed and represented in a way that captures its important features.

Another thing you can try is augmenting the “search text” you are using to retrieve similar content. One way you can do this is using the HyDE method.

Good luck with your ChatGPT,
Brian :palm_tree:

2 Likes
3

Thanks a lot.
I think HyDE is helpful maybe.
I will take a look at it

4

HyDE works very well for pulling up appropriate documents at the document level.

I can strongly recommend it to anyone struggling with surfacing the correct embeddings.

1 Like
5

I use OpenAI’s NodeJS library and so far have no problem using it for embeddings and searching things from the embedded documents but I don’t use pinecone. Is the result very off?

6

Can you provide some posts or video tutorial to use HyDE?

7

No, not well.
At first, I used MongoDB but the result was terrible, lol.
And I got the idea to use pinecone and the result was better but not that better.

8

This is a good resource:

Here is the original paper:

1 Like
9

Thanks a lot.
I am giving thanks to the OpenAI community.
:pray:

10

Hello! Could you share how you embedded a PDF using OpenAI/Node and storing it to Pinecone? Thanks!