I wanted to add top_k in as a parameter. I believe this does not exist in the same way as temperature or top_p for example.
So I tried adding top_k as part of a prompt, which works quite well until the questions become related to maths and the prompt becomes part of the instruction apperantly.
I know from other generative AIs that f.e. brackets or other seperators can be used to distinguishe between the prompt/role of the AI and the instruction.
I tried a few, which worked only to a small degree better than without them.
Does anybody have good experiences with setting up a prompt in a similar way?
Any ideas or leads are highly appreciated.
I am using the “text-davinci-003” and for the prompt:
prompt = st.session_state.conversation + tone_input + user_input + '(if writing code: start with a summary, then put the used language infront of the code in “”, then start with < code> and end it with < /code>) around every code. ’
So I add the conversation so far for reference which I want to change in the future, tone_input is another optional prompt f.e. “explain like I am five” and then at the end in bracktes is what I am experimenting with currently.
Could the order be an issue here maybe? As the user input is inbetween previous conversation and prompt?
Well, there are now API endpoints for 3.5-turbo and GPT-4, both models are significantly more performant and are dedicated “instruction” models, so they have a bias towards following instructions rather than simply performing a completion, worth experimenting with in the playground.