my apologies I answered your question in the wrong context,
I answered purely to point you in the right direction of the bit I saw.
Reading where you started from.
The guy pointed you at the tech resource and I think he was trying to point out that you were missing some technical fundamental
Which would be necessary to understand any recommendation, what I mean by this is that there are programing approached when interfacing to api endpoint tha can be used to limit the number of calls/token and given that we are still quite early in the tech evolutionof these technologies – wee don’t have the robust tooling that we find in other more mature spaces . So ealy adopters are on the bleeding edge and what do they do – they bleed the TLDR of that whole lot was their isn’t an easy solution to the question you asked.
After that wee need to clear up the the specific comment
I didn’t get context on the the response to you the roles + messages , I tjought you needed to know where to find the docs and that’s what my answer was worth - zero. T
he reply to you implied using those messages , to contain costs - I’m not certain what he was referring to and would love to understand actually( ?), I re3ad it as you use those messages/roles to focus the model and hence minimize inefficiency buy I would love to know if tht wasn’t whqt was meant.
There is one very obvious thing that is not too technical and that is the costs associated to each model. I f you use the models appropriately i.e. use the smallest model that is capable for your task. Don’t use Davinci capable for arbitray tasks – say a relatively simple sentiment analysis task – use curie, in fact try the other models as curiie is very capable and faster than davinci and ONE TENTH THE COST.
Gpt-4 DaVinci is incredibly good a certain tasks but if you are trying to create something similar to chatGPT its going to costr and hurt.
- Look at the model pricing and understand the differences between the models so that you apply the correct3engine for the task, don’t put a V6 on a moped – very fuel inefficient . ChatGPT i.e. 3.5 turbo is 0.1 cost of davinci. But there is a catch with that, it kind of had to be .because the reason ChatGPT can hold the conversation is because you send the whole conversation in each response, the whole conversation! Which brings us to the next point,
(btw Now the charge is per 1k token,s a token is similar to word but it isn’t a word, I use a factor of 0.6 i.e. see the cost as per 600 words,)
- Technically you need to understand how token are consumed given the different use cases. So first thing – you pay for all tokens consumed in the prompt AND you pay for all tokens generated in the response so:
You ask for a business tagline and some copy – the prompt is 300 words – you have consumed 300=+ tokens
It generates 500 words – you have consumed 500+ tokens
Together that has cost 800+ tokens
Now that is for a zero shot request – you put the the request out it responds – it serves you to request a number of variations – so 3 versions of the copy, as its cheaper that going back and forward with the same request. In fact this training couldhelp you:
ChatGPT Prompt Engineering for Developers - DeepLearning.AI
The catch to the above is to support a chat , you send older comments in the conversation in with the request so that the model has the context of the conversation . that’s a lot of tokens and if you are chatting to davinci – it’s a waste in most cases.
- Use the correct model forr the task ,
- Learn to prompt to get what you want efficiently.
- Consider your use case – do you need to have an entire conversation to generate an email
(aside I heard some idiot said anybody could have done ChatGPT before openAi, technically correct but, it would be good if that individual did not assume it didn’t come from the channel because of lack of imagination but rather lack BIG BLUE funding, we don’t have blank token cheques. and if you think you know who I mean – you do. Farkin eejit
Hope that helps, if not im happy to jump on call for a few mins to understand what you up to but I’m no guru or anything.