CHAT-GPT Search API For Document Upload

General API discussion
1

Hello,

I have a website with over 2,000 documents that I want to make searchable. I’ve tried a couple of techniques to achieve this, but I ran into some issues.

The first approach I tried was to convert the documents into text files and send along with the search query, to an completion API . However,
I found that this method failed when I had documents with more than 3,000 to 4,000 words, and I received a maximum token exception.

The second approach I tried was to convert one document into a JSONL file and upload it using a file endpoint. This method worked well,
but I don’t want to use it because it involves fine-tuning a model, and I have a large amount of data that I want to search through.

Is there any procedure to achieve that?
I am using .Net Language but i can convert it if any body knows it will be helpful.

1 Like
2

Hi @dev.umaralifarooq

You can use embeddings.

There’s a tutorial as well for a similar purpose.

1 Like
3

We also now have a specific tutorial on doing this with files: openai-cookbook/apps/file-q-and-a at main · openai/openai-cookbook · GitHub

4 Likes
4

@logankilpatrick thanks for your help, i don’t have access to download the next.js code?

5

@sps Thanks for your reply, do you have any other help because its much confusing.

6

Check my post, I think it can help you! I explained in detail how embeddings work.