How to Output JSON? Function calling or prompting?

API
1

Hi,

I am looking to be able to provide text to the model along with an associated tone, have the model rewrite the text in the provided tone, then send that data back as JSON.

I know that I can receive JSON via function calling, but to my understanding, this is moreso that GPT can parse natural language and assign parameters, then send those back so we can call a function with it. Is there no way to do what I have described?

Any help on the matter is greatly appreciated.

2

There’s a new model that follows writing instructions “gpt-3.5-turbo-instruct”.

Although it is called “instruct” it seems to favor completion more, where it continues writing where you left off.

You tell it you want a JSON, and then start with the first curly to show the AI it is now writing the JSON (then you assume that “{” when you handle the response.)

  prompt='Here are 10 pet chinchilla name ideas, in a json container, with numbered keys.\n'
  'Each json item has this form: key="<name_n>", value = "<generated_name>, '
  'where `n` is the item number, and `value` has the funny name. \n\n{',

Gets you:

1: “Fuzzbert”,
2: “Chinchilly”,
3: “Whiskers”,
4: “Cocoa”,
5: “Binky”,
6: “Squeakers”,
7: “Nibbles”,
8: “Puffball”,
9: “Wiggles”,
10: “Snuggles”
}

Fuzzbert would be proud of the AI.

Or you can just use a chat model - but there’s a good chance it chats at you instead of going right to your JSON output.

3

Thanks for your reply, I’ll try this shortly!

4

Actually no. You can use an example (one shot prompting) and give it a way to express itself when it can’t create a json because there is no data

or [ ]

Use this (works close to all the time):

    Always answer in the following json format:
    
    [{"key":"value", "something": {"otherkey":"othervalue"}] or []
    
    ----------------------------------------
    
    ONLY JSON IS ALLOWED as an answer. No explanation or other text is allowed.
5

Example “completion” code I employed:


import openai
openai.api_key = key
r = openai.Completion.create(
  model="gpt-3.5-turbo-instruct",
  prompt='Here are 10 pet chinchilla name ideas, in a json container, with numbered keys.\n'
  'Each json item has this form: key="<name_n>", value = "<generated_name>, '
  'where `n` is the item number, and `value` has the funny name. \n\n{',
  max_tokens=500,
  top_p=0.5
)
print(r['choices'][0]['text'])

It took a bit more tweaking to name my pet…

  prompt='''

Give 10 pet chinchilla name ideas, in a json container in array, with numbered keys.

Always answer in the following json format:
    
{"names":
    {"name_1":"<name>"},
    {"name_2":"<name>"},
    ...
    
----------------------------------------
    
ONLY JSON IS ALLOWED as an answer. No explanation or other text is allowed.

  '''.strip()

{“names”:
{“name_1”:“Coco”},
{“name_2”:“Buddy”},
{“name_3”:“Luna”},
{“name_4”:“Oreo”},
{“name_5”:“Charlie”},
{“name_6”:“Daisy”},
{“name_7”:“Gizmo”},
{“name_8”:“Hazel”},
{“name_9”:“Milo”},
{“name_10”:“Pepper”}
}

1 Like
6

Thanks, this is what I’m doing currently, but my hope is to 100% gate it to JSON so I don’t have to handle as many edge cases.

1 Like
7

I have implemented a retry count and let it retry after a minute (because it may run into rate limits - which you should eleminate as well).
And I am doing a json validation which basically increases the retry count and then saves the result into a database where i can look up what happened (it will save each type of exception as well together with response and prompt).

But in 3000 runs I barely see 1 or 2 retries at all because of wrong json format.

You may as well use a regular expression to parse out the json.

Or and that might be a little bit tricky:

you can ask it for multiple single json files.

{"name": "<name>"}
---|---
{"name": "<name>"}
---|---
{"name": "<name>"}
---|---
{"name": "<name>"}
---|---
{"name": "<name>"}
---|---
{"name": "<name>"}

This way you can even use streaming.

In your stream you have to look for a sequence of —|— so you know you have the next json coming in.

8

I am actually doing it like this…

Always answer in the following json format where “n” means “name”:

{
"names": [ 
   {"n":"<name>"},...
 ]
}

or

{[]}

will give you an array to loop over and I was hoping to reduce tokens by using the short version in key names with the mapping (didn’t check the difference) and if it can’t find any names you still get a valid json and check for the empty array later and give another exception for that - or I mean could also mean there is no result - which is still a valid result.

I am going to steal your

<name>

notation. Really like that.