don’t misunderstand this as bashing. This is a support offer from a business case perspective highlighting an issue I found.
I’m honestly trying to find a way to build a business on top of the amazing capabilities of OpenAI
my playground test with a simple domain specific conversational
text -to-> structure use case lead to cost escalations that would be cheaper to hire a human assistant.
I strongly doubt that this was the intention of the Assistant API.
Some numbers to have an impression of the Costs to Value ratio:
Assistant Instruction: ~200 Tokens
One Function: ~410 Tokens
Executed BUC Requests: 10 (+7 repetitions) [BUC]=Business Use Case = Value
Avg BUC Request size: ~10 Tokens
Avg function result size: 2 x ~300 Tokens (~100 Tokens per result, but 1 BUC request can lead to 1-4 results in avg. And 2x because I get them once as a result and use them again for the final reply )
To allow learning, I explain domain concepts:
Explained 7 concepts: ~1200 Tokens / Concept
Had ~15 requests for clarification and communication preferences: ~50 Tokens / Request
Had 5 general queries that also count as value added BUCs, while 3 were successful.
(I am not counting the produced Tokens and required Memory tokens, cause this is not in my hands and mainly efficiency topic for the Assistant API and Models applied to my requests.)
Bottom line is:
I paid ~$20 for 10 + 3 solved value added BUC transactions. => ~$1 / BUC transaction.
Sounds not bad, does it?
Well, since the Value is realized only when the Task is executed, these costs are only for getting the structured input to automate the execution in the background. So there are quite some more costs getting on Top (Compute, Infrastructure, general Biz costs sales/marketing/operations/biz. admin). All together, we might end up with $3 per BUC transaction.
So the assistant causing 30% of the costs is not so nice any longer.
But hey, it’s most probable the key enabler for the BUC, so maybe still ok. right?
Now, let’s see how much I would have to charge my customers for this BUC to make sense:
In the targeted BUC we expect to have ~20 BUC / Transactions per day
=> ~$60/Day => 20 Working Days / Month => $1200 / Month / per User
Being a nice business person asking for 10% revenue, I’d need to charge ~$1320 / Month / User for automating 1 Task.
Let’s not forget the value for the User: In the specific domain I’m talking about, this task is currently costing the User ~5 min per transaction. So we are saving the User ~1,5h a Day.
Considering the avg. of $45 / h Brutto (its Europe) => savings of ~$67 / Day
=> $1340 / Month.
The User would win $20/Month thanks to our AI solution. Yeeey ;o)
But wait!.. the User could hire a human 1,5h x 20 Days= 30h / Month = ~20% of a full time job.
The required qualification would mean (in Europe) at its best ~$3000 / Month for full time.
For 20% part-time this would mean ~$600/Month. Correlation is not linear, so to be fair, you’d end up with $800/Month.
And no matter what a Human will stay more capable than AI assistants for a while, means this human will be able to learn to execute more than this task and will pretty sure be able to cover more after a while cause the 20 BUC transactions / Day is only an average, so there will be times of no workload allowing for other tasks.
BUT wait! … it gets even better.
How about you setup a business where you employ 1 Human Assistant to execute this task. Because the interface would be the same. The user chats or calls the service, formulate his request and the task is executed. Conversational capabilities included ;o)
Consolidating this type of admin tasks can easily lead to one Human Assistant to server 20 users. And even calculating with 20 users per Assistant and a monthly Human Assistant costs of $4000/Month.
Let’s not forget about the other Biz Costs ( Sales, Marketing, … ) and someone who has to take care of this as well. Since it’s still a mini Biz. Setup count with 20-30K / Month + $4000/Month per Human Assistant.
You’d have to scale the business to exceed the breakeven point where revenue covers the Biz. Ops costs, so revenue can start to happen.
So how much would you have to charge your customers?
Well it’s still ($4000 / 20 Cust./ H Assistant ) + 10% Revenue = ~$220 / Month
And that is way less than $1340/Month.
We could conclude that this is just a Not feasible BUC.
BUT hey, it’s exactly what we target with this kind of AI: Automate mundane tasks of low to medium complexity.
Dear OpenAI Tech. and/or Sales Team I would love to further discuss this and help elaborate how to make this type of BUCs feasible, because I’m convinced that its possible, and I’m also convinced that Customers will not accept to pay more than $0.05/transaction for this kind of service (Big companies with 10k+ employees and millions of transactions will not go beyond $0.001/Transaction + max. cap).