I have the following scenario.
I want to generate a comment for a student answer to a question.
The question could be from different subjects such as math or history.
Depending on the subject I would like to use a different prompt.
I want the LLM to make the decision which subject it is and thus chose the prompt based on the decision. If I don’t have a specific prompt template for a subject, it should just fall back to a generic template I will create.
How can I achieve this?
Would function calling be the right way?
1 Like
I would write a “subject evaluator” AI prompt that gets run first. You could provide a list of five numbered topics of specialization that the AI should delegate the job to. Then, with structured output and an enum field, you could make the AI only write one of five distinct strings, or the index number.
With such a structured output, you could also instruct for more to be produced, like a clear rewriting of the prompt instructions, breaking down what it says and what needs to be performed: a planning text prompt for the next specialized model to consume.
4 Likes
Sounds good, I was initially also thinking about that. But then got hooked on the function calling idea with a function connecting to a subject template. So, I would have a function per subject.
How does your two step approach compare? I guess for one the subject evaluator approach uses fewer tokens due to not having the function calling stuff. I further guess subject accuracy wise, the approaches are the same.
What do you think? thanks!
@_j 's solution is a bit more accurate but 2 step.
A JSON request to return both prompt and type in set format or a text request to return in comma delim would eliminate a request and speed up the interaction.
[‘Type’=>‘Subject’, ‘Prompt’=>‘Prompt To Complete Task’]
“Subject”, “Prompt To Complete Task”
You could have the script return Boolean 0 on fail.
On 0 return your local default.
I would expect that sending the data back for checking would be a better use statistically for a second request.
In fact I use this method for multiple records at a time with few problems.
2 Likes
Using a tool is also a two-step approach. You are basically giving the AI a different style to write its output. Then you must make a new API call anyway - and if you don’t abandon the chat, you have to provide tool output and tool return to the chat.
Tools are a skill: the AI can call on them when they are useful.
response_format is mandatory: it has to write in the output format, always giving you the chosen “specialist”.
The only case where tool would make sense is where you want the chat to continue normally except in special situations. For example, a tool, A specialized AI that is smarter at science than you will take over when you send “topic”:“science” to this function…
1 Like