Okay so there are couple of things to consider.
-
Temperature. I gradually increased the temperature from 0 to 0.1 and then to 0.2 and so on. Higher values (closer to 1) make output more diverse and sometimes random, while lower values (closer to 0) make the model more deterministic and focused. Taken directly from ChatGPT 4.
-
Prompt optimization : Whatever prompt you write, submit that to GPT4 (not 3.5) and ask it to optimize it for you.
-
Feedback Prompting or supplemental prompts : The first response of the prompt need to be submitted back to the APIs, with a supplemental prompt that asks it to generate a formatted report. So there are total 2 prompts, fired one after the other both as a role of a user. I am not sure what difference does it make but as per the docs, I am using user.
The reason for this is that any structured data will require parsing and computational power of gpt3.5, which gives out an unstructured textual report. From unstructured the supplemental prompt can give a formatted structured JSON report, which I originally intended.
Hope this helps anyone who is struggling with the output.