Is it possible to re-use same system message for all API chat completions? if I have a rather large system message (topics for topic modelling) asking it to choose the optimal one, it can get somewhat costly sending the same large system message on each request.
Unfortunately not, each completion request is discrete.
System prompt are also the least of your potential concerns when you start adding RAG tool prompts 
The good news is that costs tend to be reduced over time.
1 Like
If you have a large system prompt required for a small input and output, that can be a case for fine-tuning. If you can train a gpt-3.5 AI by learning examples to perform the task successfully with minimum system prompt, then that can eventually become a savings after paying for the training and still paying 3x as much for the inference.
2 Likes