Is it wise to develop complex services around the GPTs ecosystem? Feeling confused

Yes.
Everything depends on whether GPTs is a user-friendly way to use AI.
Let’s wait and see in the GPTs Store.

That’s a good way to put it. To further illustrate this point, let’s think about the assistants API and what it really is. It’s designed to abstract away the complexities of managing conversation chains and context windows, but for experienced developers, it misses the mark.

All it seems to do is shift the conversation chain management to OpenAI, resulting in less control for the developer. Its current form is essentially limited, likely a rushed job for Dev Day, lacking significant functionality. A glaring issue is not being able to manage the conversation chain’s length, potentially stretching to 128,000 tokens for GPT-4 Turbo.

Every interaction involves resending all input tokens, leading to high costs for simple queries. Additionally, there’s widespread confusion about the API’s pricing and token usage.

Moderately experienced developers, familiar with creating assistants using OpenAI’s API, will immediately notice these drawbacks. It looks like the API targets hobbyists or less experienced developers who want a quick setup, not suitable for serious developers or companies looking for in-depth control. In testing, it shows errors and limitations not present in the standard API.

As a developer, I value precise control over the context window, a feature absent in the Assistants API. It feels half-baked, possibly released prematurely for Dev Day. The concept of a “stateless API” sounds revolutionary in theory but, the models are stateless so that’s not changing anytime soon, in practice, it means giving up control to a less efficient system. The lack of support for streaming is a notable oversight, given the current expectations by users set by services like ChatGPT. In its current state, it’s hard to see it being used by any serious developer.

2 Likes

@NormanNormal Thank you for such a detailed response. Indeed, everything seems very hasty.
Perhaps they are intentionally or unintentionally targeting GPTs and the Assistant API at “hobbyists or less experienced developers.”
Continue to observe!

1 Like