How can I access the GPT Store data such as the names of public GPTs and their descriptions?

It seems to me that people have been able to build GPTs that access data from the GPT store using an API.

How can I access the GPT Store data such as the names of public GPTs and their descriptions?

They don’t.

They’re using data scraped from the web, including Google and Bing searches.

Hmm… I definitely agree some are doing only that and their custom GPTs have fairly unreliable output , but for some of them it can’t just be that they are prompting the GPT to search Bing. The reason I say that is because they are consistently able to guarantee that info about only existing GPTs are provided and with viable links, I’ve tried prompting every which way with all sorts of constraints and no matter what half the time the results are either GPTs that don’t exist in the store or at the very least fake links. I thought maybe they were storing and verifying that info independently on their end.

I never said they were. I said they were scraping Google and Bing search results to find GPTs.

Like this,.

https://www.google.com/search?q=site%3Ahttps%3A%2F%2Fchat.openai.com%2Fg

Interesting, thanks for sharing that example. I actually tried to implement that url search directly as a constraint in my original prompts (which didn’t work) but not as part of my own web scrape.

One last question, I’m not too familiar with all the intricacies of web scraping (currently taking python courses that will eventually cover it), but are you suggesting that:

  1. It is a fairly doable process to scrape every single existing GPT and then you can just ask GPT to look directly at the data you’ve collected for it?

Or

  1. Is it more so that these individuals put a lot of work into to some pretty sophisticated web scraping models considering it’s able to detect the correct GPTs based on the user’s prompt better than ChatGPT prompting is able to on its own?

Also, thanks for taking the time to answer.

Stop. Prompting. The. Model.

The model doesn’t know anything about the GPT store.

What in the world are you even trying to do?

This question arose because I was trying to get practice and familiarize myself with how to create good custom GPTs for the GPT store, so that I’ll have a better understanding of what is possible to build out and how to do it when it comes to my own ideas.

I chose to practice by first recreating all those custom GPTs people made with the intended purpose of helping a user search the GPT store for a GPT based on the need they described.

Unfortunately, the instructions I gave the custom GPT totally failed no matter how I tried to structure it (like I said it often recommended non existent GPTs, invalid urls, and couldn’t stick to the script). There are some published GPT Store searchers/finders/etc. that are equally ineffective, but a few people managed to create ones that never make an error, always on topic, always relevant existing GPTs, always correct links, and so I thought this must mean they’re doing something beyond good “prompt engineering”, that’s what led me to post this question.

I thought your answer implied that they are in fact doing something different, and it’s related to web scraping which led to the final question I asked in my previous post.

To recreate those GPTs you would need to give them knowledge of the tens of thousands of GPTs, you need to collect this data and create some knowledge files to contain it all.

1 Like

I know how to create databases / knowledge files, but can something like Python really scrape the tens of thousands of GPTs by asking it to grab every english GPT that starts with “chat.openai. com/g” or is there a limit to how much it can handle?

It can do whatever you want to make it do. A couple of weeks after custom GPTs were announced I was able to find ~35,000 GPTs in a few hours of searching.

Go nuts, do whatever you want.

1 Like

Hi, @elmstedt Thanks for your reply. Can I know how you compose the website address? heuristically or any patterns that we can follow?

This will do it. Feel free to add any other query parameters to diversify the search to find more.

https://www.google.com/search?q=site%3Ahttps%3A%2F%2Fchat.openai.com%2Fg%2F

1 Like