How to restrict API Assistants to only answer based on provided information's

I have uploaded file and added doc content.

$systemPrompt = "Information = “.$content.”. "Create 2 MCQ questions based on given information. The resulting JSON object should be in this format: ";
$userPrompt = “{‘questionArray’:[{ ‘questionText’:‘Question text here’, ‘options’:[ {‘text’:‘’,‘isCorrect’:true}, {‘text’:‘’,‘isCorrect’:false}, {‘text’:‘’,‘isCorrect’:false}, {‘text’:‘’,‘isCorrect’:false} ], ‘correctFeedback’:‘’, ‘inCorrectFeedback’:‘’ }]}”;

i am using ‘model’ => ‘gpt-4-turbo-preview’,

Sometime it is using some word/text in question which is not present in the information provided.

If the AI seems to be reading back text that was retrieved, but deviates in word use, substituting synonyms or deviations, that is likely an effect of sampling, word probability based on AI certainty, and lack of temperature to control this.

If I told the AI to repeat back my first paragraph above, it may have calculated that “deviations” was 95% the token to emit. However, that allows a 5% chance of an alternate token being selected, from which the AI then has to continue writing in a way that makes sense or try to get back on track. That process continues for each token being generated.

Answering not from existing pretrained knowledge, but only the information provided in files requires instructions to that effect that keep the AI closed-domain. You can tell the AI what it doesn’t know and what it can’t answer except by seeing the information in files.

The AI actually having the document to answer from relies on the ability of the search function and lookup matching quality that operates on document retrieval, not something you can tune.

2 Likes