For my organization, I want to use few-shot examples in my prompting with structured output.
I did go through this thread:
https://community.openai.com/t/few-shot-prompting-with-structured-outputs/1045058
But it wasn’t very helpful with what I am trying to achieve.
After some digging, I also encountered a way where we can use function calling with structured outputs (which is in beta, currently, btw)
I am not intending to use function/tool calling to impose a strict behaviour.
Primarily because if we intend to switch to a local model, they don’t support took calling.
Is there any other work-around for this?
UPDATE: I mentioned that local models don’t support tool calling. That was wrong of me to mention. The primary reason is that we would want to use Pydantic with structured outputs. Hence, the requirements of ingesting few-shot examples with structured outputs.
2 Likes
I am giving it examples of json it should produce and do a json schema check on the response…
your last sentences after example structures in the prompt could be
1st. {example json}
2nd…
3rd…
Create that from this data:
/=======================
[some input data]
/=======================
Only answer with the requested json!
Start your response with { and end it with }
— which does not prevent hallucinations
Hi @rpushpak and welcome to the community!
Just a little note on this statement:
Primarily because if we intend to switch to a local model, they don’t support took calling.
There are actually several packages out there that allow you to implement structured outputs and function calling on open weight / local models. One solid example is the outlines package.
1 Like
Hi @platypus
Thanks for the welcome and the reply.
I mentioned it wrongly and have updated my post. The primary reason is that we would want to use Pydantic with structured outputs.