I was experimenting with structured and unstructured APIs for creative writing in non-english languages. This is an observation I thought might interest somebody.
Regardless how I set the temperature and presence_penalty in the structured output API the output is inherently less creative. If I only need a text and my output format looks something like:
class ResponseFormat(BaseModel):
response: str
model writing will always be more conservative and less creative than non-structured API, and by a significant margin. One may ask why not simply use an unstructured API instead for such cases.
Convenience of using structured output is higher. It allows to e.g insure model does not add extra comments or formattings without explicit instructions. It also allows to split automatically the response if I ask to generate several examples and in this case the format will be something like examples: list[str]. It also allows to have a single simple function to query the model.
Currently structured api method does not accept a None or empty or str in the response_format argument. So I cannot simply parameter this method to achieve non-keyword string output.
This behaviour can also imply some more fundamental differences in what tasks each type of API is the most suited for and I wish I knew about these differences earlier.
This is not a bug per se, but it would be better and more logical to have similar output distributions for non-structured call and structured call with a single simple ``response: str` keyword in the output format.
Just for reference, my structured output usage example:
response = await async_client.beta.chat.completions.parse(...
Unstructured api usage example:
response = await async_client.chat.completions.create(...
Current python openai lib 1.65.3.