Custom GPT performs differently in preview than when live

Anyone had this issue?

I’ve built a GPT that connects to the Spotify API. It gets my user id, creates a playlist, searches for track URI and adds the tracks to the playlist.

It works flawlessly in preview mode. But when I push it live, it inevitably struggles, get stuck in search loops, and adds the wrong songs to the playlist.

Anyone run into this? Any ideas on how I can get more consistent performance?

2 Likes

I observed the same problems (except the “adds the wrong songs” part).

If the live performance problem isn’t fixed in time, I don’t believe the launch of the GPT Store will succeed.

I’ve experienced the same (not related to Spotify APIs), preview works much better then the published version of my custom GPTs with custom actions.

2 Likes

Maybe this issue of inconsistent performance is related to the stability of custom GPTs over time. I am training one and recently I noted that it had forgotten about the existence of one of the documents that I had uploaded in its knowledge base. It seems the custom GPT forget things, and also I have not found information on best practices on how to tune or train them.

Hi @missjenny ! Sorry for the offtopic but I am trying to build a similar GPT to create playlists for my study sessions but I am struggling handling Spotify authentication. Would you please share if possible any resource you have used to learn how to set it up?

Thanks a lot!!

Lucas.

I have the same experience with the custom GPT. Performs extremely bad when published and forgets the existence of knowledge files. I wish to have a local model that I can train with the knowledge base and not deal with so many errors with custom GPT.