I do research for a medical center project. We are developing ideas for the healthcare market and made an interesting chatbot with GPT-3 5-Turbo. So when I had the chance to try the new GPT-4 API, I directly did. The way GPT-4 handles the system details is way better. It really sticks to the role given. Absolutely amazing results. Unfortunately, I realized how much more expensive the GPT-4 API is. 
Maybe I did something wrong, but tokens added up quickly. So beware when testing your tools on scale. Other than that, it’s the best thing since sliced bread… Cheers!
2 Likes
Hah. Yeah, I noticed last night too with several hours on GPT-4…
I’m sure the price will come down eventually… as the earlier models have… the ChatGPT-turbo costs kinda lets us know as devs that’s what they want us mostly using I think haha…
2 Likes
You probably didn’t do anything wrong, but you may not have done enough to shape your bot to optimize for costs.
In the future – actually now – there will be consultants who specialize in designing systems that use financial boundaries to shape solution requirements and architecture. I’ve been approached by three clients seeking exactly this sort of assistance. My accounting and finance background may actually come in handy after all these years of trying to forget that crap.
We tend to assume that prompts and parameters constitute the limit for controlling costs with GPT. But they’re just the direct elements in the cost envelope. There are many ways to lower AI costs while increasing performance. GPTIndex, embeddings, and fine-tuned models constitute the short list of possibilities.
1 Like
Yea, I can’t believe there aren’t more people talking about this.
If you do the math, gpt-4 is a 2900% increase over gpt-3.5-turbo. That’s NUTS!
Where can we formally petition the powers that be to bring those costs down asap? Those of us who just made adjustments to product prices have to make them all over again, and it’s really going to hurt our pricing models.
3 Likes
Yep, insane. Unfortunately, for now, it’s unusable for my use case.
But the potential is awesome.