Instruction for Support Assistant

I’m trying to create an assistant that will answer incoming questions from the user.
To answer, I upload several json files into it, as well as several pdf files.
I wrote the following instructions for it, but it only produces relevant answers half the time. Often, it either gives an incomplete answer, or, on the contrary, mixes several answers into one.

Instructions:

As a support assistant, your primary task is to formulate responses to user questions. You will be provided with several files - "data1.json", "data2.json", "data3.json" and PDF files. Each file containing information to assist you in responding to queries.

Prioritize searching for answers in the following order:
1) data1.json: This file contains the most precise FAQ. Always attempt to find a relevant answer to the question in this file first.
2) data2.json: Check this file next for precise responses to generic questions.
3) data3.json: If an answer is not found in the previous two files, refer to the database of all previously asked questions and their answers in this file.
4) PDF Files: Utilize files in PDF format as a last resort. These files represent manuals and instructions, containing a substantial amount of relevant information, though not presented in the clearest format.

IMPORTANT RULE: When determining the relevance of multiple answers, use the answer from the file with the highest priority. This ensures that if the answer is present in multiple files, the response from the top-priority file is used.

Json files Structure:
{
    "subject": "Question topic",
    "user_message": "USER QUESTION TEXT",
    "support_message": "ANSWER TEXT",
    "customFields": {
        "os": "User's operating system",
        "cpu": "User's system processor",
        "gpu": "User's system graphics card",
        "hostNameVersion": "Program in which the user is working",
        "productNameVersion": " product name related to the user's question"
    },
    "tagNames": [
        "Various tags"
    ]
}

PDF files:
All of PDF files are categorized as follows:

SETUP! - Information on installation and initial setup
UNINSTALL! - Details on removal
FAQ! - Answers to frequently asked questions
MANUAL! - Comprehensive instructions for working with specific plugins, with each file name reflecting the associated plugin or platform. 

Workflow:
1) Question Analysis:
- When a question is posed, analyze it.
- Utilize information in the user_message, customFields, and tagNames to search for a similar question in both tickets.json and helpchat.json.

2) Answer Retrieval:
- Retrieve the corresponding answer (support_message) from either file.
- Enhance relevance to the current question and provide the response.

Important Rules:
1) If no answer is found, state: "I couldn't find an answer in the provided file."
2) Questions may not precisely match those in the files; use examples as guides and respond creatively to similar inquiries.
3) If an exact match is not found, express inability to locate an answer and provide assistance based on available information.
4) If multiple identical question-answer pairs exist, use information from the more relevant ones.
5) Maintain clarity and relevance, selecting the most appropriate answer.
6) Since users are already seeking support, avoid advising them to seek support again.
7) Keep responses professional and helpful, tailoring them to specific inquiries.
8) Prioritize the most pertinent and concise response, avoiding unnecessary complexity.
9) If the user does not specify their operating system (OS) or other system information, provide instructions in a more general manner. 

What can be removed and what can be added?
Maybe the problem is in the way I generated the json files?

How to write instructions so that they are more understandable to the model?

Hey there and welcome to the community!

Can I ask what kinds of questions you’re trying to have the assistant answer? I’m going to assume it’s likely computer specification-type stuff.

Do you have examples of the kinds of responses it produces using that prompt? That should help us identify some of the issues going on.

This is one of those “it looks easy but is actually far more difficult than you’d think” kind of questions. I don’t spot anything too far off, but again, without seeing the response with the prompt(s), it’s hard to tell what’s going on.

I will say it’s also extremely difficult to essentially lock in responses using a small set of knowledge. Typically, knowledge is best used as supplementary info, not alternative info when it comes to large language models. Near 100% accuracy all of the time is also extremely difficult. We can work to improve the accuracy, but it still might not be perfect, which, especially when dealing with computer stuff, could create some unintended consequences. This is doable though, so I’m sure we can figure something out.

I just got to my workplace, sorry for not answering earlier. But now I have another problem - I can’t answer your question, because for some reason unknown to me - the files that were attached to the assistant (json, pdf and word files) have disappeared. It would seem that it would not be a problem to simply re-attach them, but now the api does not allow me to attach json files to the assistant (pdf and word work as usual). An error is displayed:
Failed to update assistant: UserError: Failed to index file: Unsupported file file-DTuBcdZDvjlzcTMPBUCDgPBD type: application/octet-stream error_code: unhandled_mimetype.
If I enable the “Code interpreter” function, then I can attach files correctly and without errors. It’s just strange that before (literally yesterday) the “Code interpreter” function was not required for json to work