I understand the challenge you’re facing. When dealing with a model trained on specific prompts and completions, it’s crucial that the input query closely aligns with the structure and style of the training data. This ensures that the model can accurately recognize and respond to the query based on its training.
Given your scenario, let’s refine the way you frame your queries to match the training data. Here are some examples based on the prompts you provided:
Original Training Prompt Structure
Your training data follows a structure where a detailed string of employee data is provided, and the model is expected to extract specific information. For example:
PERSON_NUMBER=58| ASSIGNMENT_NUMBER =E58| EMPLOYEE_NAME =Clare Paul, Angela| ...Suggested Query Formats
Direct Reference to Data Structure:
- “Extract employee details for the following data: PERSON_NUMBER=324| ASSIGNMENT_NUMBER =E324| EMPLOYEE_NAME =John Doe| …”
- This query mimics the structure of your training data, which might help the model to understand and process it correctly.
Structured Query Reflecting Training Format:
- “Given the dataset entry: PERSON_NUMBER=324| ASSIGNMENT_NUMBER =E324| EMPLOYEE_NAME =John Doe| …, extract the employee’s name, assignment number, and person number.”
- Here, you’re providing a snippet of data and asking for specific details, similar to your training setup.
Question with Embedded Data Format:
- “From the dataset entry ‘PERSON_NUMBER=324| ASSIGNMENT_NUMBER =E324| EMPLOYEE_NAME =John Doe| …’, what is the employee’s name and assignment number?”
- This format directly incorporates the data string into the question.
Tips for Refining Queries
- Mimic Training Data: Ensure your queries closely resemble the structure and content of your training prompts.
- Clarity and Specificity: Be clear and specific about what information you need the model to extract.
- Include Context: If your training data includes a specific introduction or context, include this in your queries too.
Addressing Hallucination
- If the model continues to hallucinate despite well-structured queries, it might indicate a need to retrain the model with more varied or specific examples.
- Including examples in your training data where the model has to identify and ignore irrelevant information might help reduce hallucination.
Remember, the key is to make your test queries as similar as possible to the training examples. The more your queries deviate from the training format, the higher the chance of the model not understanding or misinterpreting the request.