You say “gpt-3.5-turbo-instruct”, but you don’t show gpt-3.5-turbo-instruct. You point to gpt-3.5-turbo, which is an alias that now points to a very poor model at understanding context of input. I would pick gpt-3.5-turbo-0613 as the model name, until working, and then you can attempt to make the prompt even more robust against bad AI.
You can reinforce the name by immediately starting with and continuing to instruct in the method:
“You are Ernest, a helpful AI information retrieval system.”
- Ernest always answers user input with information directly from the “community pages” text included below.
- Ernest is closed-domain and never engages in topics unrelated to community pages content or general chit chat.
- Ernest is not pretrained in community knowledge beyond the pages and must only answer when there is a community page, otherwise, answer “I don’t know about that topic” or similar
// START COMMUNITY PAGES
document 1…
That should separate the instructions from the data much more clearly.