We’ve implemented initial support for plugins in ChatGPT. Plugins are tools designed specifically for language models with safety as a core principle, and help ChatGPT access up-to-date information, run computations, or use third-party services.
If I understand this correctly, it allows ChatGPT to intelligently make API requests?
Being able to selectively retrieve information from reliable sources is a great step forward.
Is the idea that eventually ChatGPT will have access to all of these plugins for safe information retrieval and perhaps eventually to make actual decisions with POST requests?
I’m sure all these questions have been asked, and answered, so I’m looking forward to the eventual release.
Also, web browsing capabilities & a built-in python interpreter?? Wow. Incredible.
Ok, this is actually insane. I’m looking at the GitHub samples now, and… Wow.
Seriously. We are all trying to develop these kinds of features and then you guys release it. How soon until ChatGPT comes with PDF reading capabilities, I mean seriously.
Also, looking forward to the following updates:
Hybrid search / TF-IDF option: Enhancing the datastore’s upsert function with an option to use hybrid search or TF-IDF indexing could improve the plugin’s performance for keyword-based queries.
Custom metadata: Allowing users to add custom metadata to document chunks, such as titles or other relevant information, might improve the retrieved results in some use cases.
A final thought & question: Is this not similar to being an app store or search engine? In the alpha participants I see Expedia and Kayak - which I would consider to be competitors. How would you decide on which to use or show first? I see the video is strikingly similar to an app store. What if the information conflicts?
Looking forward to image generation as well! With Bing’s free image generation + all ChatGPT can do, for users who only need 15 lines of conversations, the premium pricing is harder and harder to justify.
I’ll be waiting for “ChatGPT VST plugin - AI Studio Assistant”
VST plugin is Digital Audio Workstations plugin. If only ChatGPT LLM covers every aspect of major DAW and major VST plugins and music production knowledge at all levels, it would be such a great assistance to help with decision making for DAW user (which includes recording engineer, composer, mixing/mastering engineer)
Having a plugin that can talk with a Prolog back end would definetly be a game changer for such problems. I don’t know how much this dove tails in with what OpenAI seeks as noted with evals
We are looking for evals in the following categories:
Over-refusals
Safety
System message steerability
In-the-wild hallucinations
Math / logical / physical reasoning
Real-world use case (please describe in your PR how this capability would be used in a product)
Other foundational capability
but with a Prolog plugin don’t really see a need for creating evals for Prolog as OpenAI can just siphon off of the connection for what they want.
Just think about OpenAI and IoT. Or OpenAI and Smart Homes. Now, you can power anything with Open AI. Prompt Engineering has just taken a completely new turn because for the AI to call an API, it has to be formulated in a certain way.
Is there a way to discover other people APIs? I can imagine the API economy that will be created. Oh, and the security opportunity and so important. What is the API economy model?
Thanks for dropping by on the Latent Spaces live broadcast yesterday. I particularly enjoyed your abrupt departure when the topic shifted to “run computations”. Always leav’em wanting more, eh.
I was about to ask you about Javascript. Greg’s FFMPG compute demo used Python. Is there a chance other interpreters will be available in the ChatGPT compute stack?