I love the Assistants API. I am building the capability into my SAAS in order to provide users the ability to interact with their data i.e. sales, customers, etc. The functions feature works like a charm and a user can easily interact with their data using human language, i.e. “find the sale for customer bob jones”.
However, it is only truly accurate with GPT-4 and the pricing for GPT-4 as of this post makes Assistants prohibitive at scale for a user of my service.
Conducting simple interactions with data and running functions is not cheap, especially if the relevant item the user is interacting with isn’t small in size. I understand and love the concept of threads, however the compounding nature of a threads’ token usage makes matters worse.
I have looked into file store, but it does not fit for my case due to data being in JSON, the number of files needed (a file per sale, per customer, etc.) and the need for a file to be processed before it can be used (a chat must first wait until the file is done).
I hope OpenAI plans to considerably reduce pricing for GPT-4. Its really the only way to make assistants cost effective for most use cases. Latency is a whole other issue.
I am in the same boat as you. For me GPT 4 is ready for prime time in the way it handles conversations, follows instructions and provides an all around phenomenal user experience. Plus the functions capabilities of calling an external API is a game changer. There is simply nothing close now. I won’t even roll out my SAAS with 3.5 because it provides so much of a lesser experience than I want to offer. The way I’m handling the costly pricing for now is pitching it to my best customers as a “premium service”, “do you want the smartest AI technology available?” Those that recognize value and productivity will pay up for it no doubt. I have clients generating letters from complex 300 page PDF’s that would normally take hours for someone to work on, and now is done in a minute. So all in all, when you factor that in it doesn’t seem so bad. I’m just putting lots of padding on the margin to cover myself.
The pricing will make it prohibitive for someone that wants to spend $5 on a SAAS and have access to this type of tech. I’m not sure that’s the market I’m going for anyway.
I’m holding out and continuing my development because it will no doubt get cheaper and better as newer models roll out and then I’ll be ready.
