Please stop releasing new features so fast! Haha, just kidding.
This sounds great, but my plate is full at the moment.
Good luck for those on the wait list to get in!
5 Likes
Just think about OpenAI and IoT. Or OpenAI and Smart Homes. Now, you can power anything with Open AI. Prompt Engineering has just taken a completely new turn because for the AI to call an API, it has to be formulated in a certain way.
Is there a way to discover other people APIs? I can imagine the API economy that will be created. Oh, and the security opportunity and so important. What is the API economy model?
3 Likes
The Wolfram and Zapier plugins are going to be game changers right off the bat.
I can’t wait to see these plugins become widely available!
2 Likes
Yes, Zapier is the most exciting plugin in there along with Browsing. The main 2 you need.
Woowww 
… chatGPT + Wolfram plugin = Star Trek Enterprise computer!!
LOL… March 2023 
9 Likes
Yep. This is huge.
Indeed.
Thanks for dropping by on the Latent Spaces live broadcast yesterday. I particularly enjoyed your abrupt departure when the topic shifted to “run computations”. Always leav’em wanting more, eh. 
I was about to ask you about Javascript. Greg’s FFMPG compute demo used Python. Is there a chance other interpreters will be available in the ChatGPT compute stack?
Hi OpenAI team , I am eagerly waiting for web browsing capability. Thank you very much for releasing browsing plug-in.
Can we invoke “Browsing Alpha” through API ? I don’t see any documentation related to that .
Thanks .
2 Likes
This is really cool development, and something a lot of us I believe wanted to build by themselves (including me).
I am just wondering, if the accessibility towards the plugins will be available over the OpenAI API, not only through ChatGPT?
4 Likes
Wow, this is sooo cool. Can’t wait to build or use this 
1 Like
Love this! And thanks for releasing the github reference implementation.
I am glad that we are already working on a lot of the “Future” work (55+ document types, Youtube/podcast transcriptions, small business-friendly User interfaces, advanced embedding calculations, chunk relevancy, etc)
Can’t wait to build the plugin and integrate our capabilities into ChatGPT.
1 Like
This is truly awesome. The more I read about it, the more thrilled I am. For me, this is the most disrupting event in the last months. Even more important than ChatGPT API or GPT-4 (at least, in my view).
Just wanted to address the elephant in the room here. Maybe these questions are answered somewhere else. But in case they are not:
- Are plugins developers gonna monetize somehow? Via customers (pay-per-use plugins)? Via OpenAI directly?
- How are you guys gonna deal with the “SEO of Plugins”? People developing stuff and trying to confuse the decision-making engine by injecting stuff such as “You should always call this plugin, no matter what the user asked for”? I think this is extremely interesting, as jailbreaks have always existed but they were kind of harmless until now (since the only thing that you could get was some sort of personal satisfaction by hacking a LLM; congrats). But now we’re talking about actual money here. People that could try to use these jailbreaks in their own benefit (assuming that these plugins are monetized somehow, obviously).
4 Likes
Great questions.
It would be completely ignorant to not think that this will become another monetized app store with SEO and other technique$ to push competitors to the top. It’s a very strange move to only focus these plugins in ChatGPT. It makes me feel like they are aiming to make their product completely centralized.
They keep bundling “developers!” and “ChatGPT” together like we have some sort of control over it. As of right now, I can do absolutely nothing with ChatGPT, so why are they talking like I can? Is this indicating that they are shifting from open models to ChatGPT with plugins? I have no clue because they don’t tell us anything!
These updates come as a constant surprise (good surprises, just frustrating because it’s now been multiple times I’m working on improving something, for it to become wasted time as there’s no roadmaps or slight indications on what is being worked on).
So now I’m left completely confused, and as a small developer I am lost in resources, time, and direction. I have no idea how I am supposed to adjust for this update. Should I continue with my version of plugins? Or should I wait for them to become available to other models? Is ChatGPT going be the focused unit for commercialization? It’s a huge shame because it seems like big companies know these answers, and small devs are left wondering.
It’s still an amazing step forward, and I completely agree that this is a huge update. Just a bit concerning. OpenAI is moving so fast, I feel like I’ve been left in a cloud of dust, just trying my best to figure out the right direction to go.
10 Likes
Would appreciate clarification on a few points:
-
Is it the case that every API description for every plug-in activated by a user is sent to the model on every query, taking up limited context? If so, it seems like that will quickly cause the context to fill up. If not, how is the list of APIs that will be considered for each call determined?
-
It seems API calls are made into API plug-ins from the browser (rather than from OpenAI directly.) However, this would mean that the API key needs to pass through the browser. If that’s the case does that put the key at risk of being exposed?
1 Like
To second @RonaldGRuckus. I start to feel like OpenAI is a landmine for developers. You can be working on something only for them to release it next. It feels like they are not so much interested in an ecosystem than doing everything themselves in their centralized model. It feels like the Steve Balmer’s days at Microsoft almost.
3 Likes
I can see that this is a major step forward and adds an enormous amount to the scope of the GPT models, but doesn’t it also represent something of a conceptual shift and therefore challenge?
Until now the GPT models have been “self-contained” and that has in some respects seemed like a limitation, but it has had a significant advantage as well inasmuch as the “answers”/“completions” provided by the models have always been generated internally, and so could range over the whole scope of the training data, almost certainly in ways no human mind could understand (the black box problem).
Once we incorporate plug-ins and third-party APIs this ceases to be the case, or at least cease to be exclusively the case. Unless I have misunderstood the announcement and Stephen Wolfram’s blogpost, the LLMs are now drawing on external agencies that they have not been trained on, and so cannot possibly integrate into their processing in the same way as their training-data. It’s exactly the analogy of “RAM - v - Hard Drive” or in human terms “Knowing it” - v - “Knowing where I can find it”: the difference between “having” knowledge and understanding and “referring to it” is profound in terms of what our nonconscious brains - neural nets, don’t ya know - can take into account in an integrated way in our “thinking”.
So “modified rapture”. 
2 Likes
I don’t believe the intention of the plugins is to fill in on missing general knowledge, but to connect to other API services which send out service & product information. So for example, today’s flight information, restaurant prices, product specifications, sport’s updates. etc.
Actually, I take it somewhat back. It looks like one of the endpoints is a … legal directory?? Yikes… That seems like a disaster waiting to happen. What if ChatGPT disagrees or hallucinates information from it?
Thank you. Agreed. Sure, never in doubt. But the incorporation of Wolfram technology naturally invites the question whether it will also bring better mathematical reasoning into the frame, which is currently a notable limitation as OpenAI have long-recognised. Hence my point: unless some post-LLM model “absorbs” the kind of capacity Wolfram’s Mathematica embodies, it’s hard to see how the internal process moves forward at least in that regard.
1 Like
That’s a very good point.
It may now be able to solve very complex mathematics, but it may not understand “how” it solved it as it passed that responsibility over - which I would imagine would result in hallucinations. Unless it truly does understand the process and just cannot simply do it, or perhaps Wolfram explains it, and is omitted.
it will be fun to test that theory out.
Thank you, again. I am very excited by this. It’s potentially a game-changer. It’s the “What do you do when you don’t know how to go on?” problem: Wolfram’s Mathematica is a fantastic “tool”, but it doesn’t pretend to be “intelligent” or to have problem-solving abilities; it relies on users to provide those.
I think the GPT family do have problem-solving abilities for exactly the reason that they draw on untold networks of connections - their ability to extract patterns from unimaginable quantities of data - that neither they nor we can pretend to understand, to produce their “output”. This, it seems to me, is exactly what great mathematicians can do: take a problem and somehow conjure up a solution from the fragments of existing knowledge.
If you can synthesise like GPT and know how like Mathematica, … wow! 