Often you need the output of the LLM to be in json format. Even when you include phrases such as “the output must be in JSON format” or variations thereof, the LLM sometimes outputs something like “here is the output in json” or “```json”. This makes the task of downstream parsing of the output of …

Did you use JSON mode? https://platform.openai.com/docs/guides/text-generation/json-mode

I wouldn’t muck about with logit bias for this, because it can taint the rest of your generation. (depending on what you’re doing, ofc) In detail: "90": 10, // token ID for "{" could motivate the model to open more sub objects than expected in certain contexts "92": 10, // token ID for "}…

We had a similar thread recently. Can confirm that using JSON mode overcomes the problem with the unwanted prefix, e.g. ```json. [image] API with 4Turbo modal not providing a clean JSON API Hello, Maybe someone here can help I have a prompt that worked well with GPT 4 a…

[image] Diet: once you’re over the initial Hump? You can be explicit in your instructions: // AI Response from assistant MANDATORY: Every single response produced after the "assistant" prompt must begin with the exact characters shown within the single-quotes: '{"function_name": "' MANDAT…

I believe you came up with this Start your response with { (or [, depending ofc) Using that as the very last line the chat model sees tends to yield pretty good results. After that, it’s mostly smooth sailing (apart from the apostrophe issue, which can be overcome by using the right schema hint…

[image] Diet: if our dear AI providers allowed us to take off the training wheels Yes, other competitive AI (ahem) allows assistant prompt completion, where you could place that JSON opening and a key directly after the internal “assistant” that is automatically added. It works a treat to wri…

Nice Tip , I developed a tool ( aipify dot co ) specifically for this issue, and now all my side projects are powered by it.

Make JSON output more likely

API

jr.2509 April 16, 2024, 4:01am 4

We had a similar thread recently. Can confirm that using JSON mode overcomes the problem with the unwanted prefix, e.g. ```json.

Topic		Replies	Views
JSON Response + logit_bias API	4	1057	March 15, 2024
There is a new Broken-JSON variant in the response API gpt-4 , api	21	1586	June 1, 2024
How to get 100% valid JSON answers? Prompting gpt-4 , gpt-35-turbo , chatgpt , api	15	11018	June 11, 2024
Valid json every time? Prompting	16	12586	March 27, 2023
Ensure JSON response format API	22	47033	January 23, 2024

Make JSON output more likely

Related topics