Using GPT 3.5 turbo for intent parsing for a custom chatbot

Prompting
, , , ,
1

I am creating a chatbot for my flatmates and me. It’s a small python script that should keep track of our shared shopping list. I want to use gpt3.5 to parse free text messages and find out if the user wants to add or remove items from the list, retrieve the list or delete the list. I am currently doing this:

    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[
            {"role": "system", "content": 'Interpret the following user input and convert it into JSON of the form {"cleaningtask": boolean, "shoppinglist":[{"action":"add"|"remove"|"list"|"clear", "items": ["string"]}]. Only return JSON. User input:'},
            {"role": "user", "content": message}
        ]
    )

It works perfect in chatGPT (also using gpt3.5), but the API gives me flaky results (is it because I’m using the system role perhaps? It adds punctuation to the end of the JSON object, or it says stuff like “Sure, here is your JSON:”.

Am I using the wrong tool for the job? Are there other simple tools that could do it better? Should I improve my prompt?

2

Welcome to the forum!

It sounds to me like you want to add another user and assistant example before the final user message. They’re still tinkering with weights on system role, I believe, so it can be a bit flaky.

I can give an example if needed.

3

I added some examples and it works better now, thanks!

1 Like
4

Good to hear! Thanks for coming back to let us know.

Hope you stick around the community. We’ve got a great bunch of people here!

5

If anyone stumbles upon this who has the same problem, this is the improved prompt that I am currently using. Please note that the input is mixed language as our chat group is primarily German. It absolutely blows me away that GPT3.5 doesn’t care that my system prompt is in English while the chat is in German.

        response = openai.ChatCompletion.create(
            model="gpt-3.5-turbo",
            messages=[
                {"role": "system", "content": 'You are a program that transforms user input to JSON of the form {"shoppinglist":[{"action": "string", "items": ["string"]}]}. Action can be "add" or "remove" or "list" or "clear". JSON ONLY. NO DISCUSSION.\n\n'},
                {"role": "user", "content": f"Hey Bergerbot, wir brauchen Karotten und Klopapier, aber keinen Allzweckreiniger mehr."},
                {"role": "assistant", "content": '{"shoppinglist":[{"action": "add", "items": ["Karotten", "Klopapier"]}, {"action": "remove", "items": ["Allzweckreiniger"]}]}'},
                {"role": "user", "content": f"Hey Bergerbot, gib mir die Einkaufsliste"},
                {"role": "assistant", "content": '{"shoppinglist":[{"action": "list", "items": []}]}'},
                {"role": "user", "content": f"Hey Bergerbot, ich habe alles eingekauft"},
                {"role": "assistant", "content": '{"shoppinglist":[{"action": "clear", "items": []}]}'},
                {"role": "user", "content": f"{message}"},
            ]
        )

I will continue to experiment more. For example, ChatGPT4 suggested I change the system prompt to:

response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=[
        {
            "role": "system",
            "content": 'You are a program that converts user input into JSON of the form {"shoppinglist":[{"action": "string", "items": ["string"]}]}. Actions: "add", "remove", "list", "clear". Provide JSON output only. Example: if a user says "add apples and oranges", your response should be \'{"shoppinglist":[{"action": "add", "items": ["apples", "oranges"]}]}\'.'
        },
        {"role": "user", "content": message},
    ]
)
1 Like