I think I read somewhere that the rambling nature of ChatGPT which many people don’t like is actually something that improves the quality of the answers, because you give it a chance to think out loud.
I want to query structured output through the API, but I’m wondering if it’s better to let it answer in free form so that it can think out aloud first, and then ask it to give the final answer in some distinct way such as wrapped into a html tag, so that it can be parsed. The structured output response_format
does not have this option though (I guess?), so I wonder if you’re making some sacrifices in response quality there.
Any thoughts?