Hi,
We are using prompts to generate some contents guided in the promts but its taking more than usual time to render. How can I improve that?
You can see if your assertion is true. Here is an independent site that regularly accesses the API and checks the generation time required.
One can also see a daily peak in time required around 7am California time - a time when all the time zones before California are awake.
The token output generation rate will be affected by the amount of input that you send. More context loaded for the AI to understand means more processing per token to generate weights and output.
1 Like