That is true, but in my case the cost of prompts is literally next to nothing. I use 15x tokens just messing with playground than all my customers together (given we currently have around 1000 users and completely free unlimited generations.
Limiting the prompt size is good and all, but with a ~1000-1200 tokens curie prompt that contains 5-6 examples it can give me 10 end results (“n: 10”) and it still cost below $0.01