How does the knowledge of custom GPT actually work

Documentation
1

How does the model actually use the files we upload? Do they just give the text to the model as context to use?

Or is it more sophisticated?

Do they embed the files or rather the text inside of them or what?

5 Likes
2

Following :eyes:

One would assume it’s the same sort of strategy to implement a RAG system:

  • split the document into chunks
  • convert the chunks to embeddings
  • when a user submits a query, convert it to embeddings
  • run a cosine similarity, finding the top X number of items
  • add the original text from those items into the prompt as context

But they could be doing something completely different, I’d like to know too! :+1:

3

I could prompt dump the system message and its tools at various sizes of retrieval data added to also see where a reported “browse data” function has been inserted to iteratively call on your uploaded data instead of it being placed completely in the context or chunked with embeddings and inserted into context as described.

However, I’m waiting for them to press “delete” on the whole project, and am not investing in a single API call.

4

So you think there’s some kind of function defined that the model knows that will load some parts of the uploaded data into the context?

Also what do you mean with them pressing “delete”? You mean that OpenAI will kill custom GPTs?

1 Like
5

Not GPTs where OpenAI is paying the bill for people that could use them as actual plugin replacements in ChatGPT Plus. That’s only an internal boondoggle for OpenAI to sort out. The load of these things going nuts on big data may be part of performance issues reported and the shutdown of new Plus subscriptions and reduction in the number of inputs a Plus user gets.

Rather Assistants, which is an API feature. And when it runs multiple loops of iteration while loading the context with untrimmable chat and filled with retrieval tokens, you pay, and you can’t find out how much you paid.

This “sorry Dave” was reported as an AI response within assistants:

"I have been provided with two files, but unfortunately, neither of these files is accessible with the myfiles_browser tool. The file IDs are:

  1. file-tbi...
  2. file-Cxr...

For further assistance with these files, you would need to provide them in a format compatible with the myfiles_browser tool

The user obviously is not at the level of writing their own functions, and yet the AI describes a # tools within assistants that is not documented.

So yes, turn it off, OpenAI. Admit failure. Stop emptying inexperienced user’s bank accounts. OpenAI knew what they were doing when devday required a new usage page that doesn’t show usage except by day, taking over a day for all the charges to show up. Get someone that writes chatbots to write your chatbots.

6

Ahhh, so you mean that the API feature “Assistant” will be deleted.

I find the “error” message you quoted from another post quite interesting. So apparently they have some function that the Model is aware of to browse files.

Well, the thing is, the assistant api is just the same as the custom gpt feature in chstgpt+. Just for devs / api calls. But as you have stated it’s impossible to know how much your gonna pay to use the “knowledge retrieval system” they have in their fixed custom gpt / assistant api.

In chatgpt+ it doesn’t matter since we don’t pay that but with the api calls that changes.

That’s why I personally would never use it, just build your own knowledge base system and just use the LLMs as you need them in your own config.

They rly could make it easier to see exactly what you pay and for what. From my tests it’s sometimes hard to preemptively calculate the cost for certain api calls with different tokens, even tough I use the latest documentation / price list for reference.

7

I did some digging and a I agree with a lot of what @_j writes, even though I still think the tool is great and hope it won’t be turned off anytime soon. But yeah, I wouldn’t recommend a public facing Assistants API for now. I had some API calls that returned 32k tokens worth of quotes of which 30k were duplicates.
A detailed write-up can be found over here and I summarized the key findings in another forum post over here.

8

I had about 5,500 of pdf pages. I loaded 10 files ranging between 6.5 GB and 8 GB and it seems to work fine. It will run slow at times and you need to reload the query but no mistakes in there response and seems to learning fine.