if you uploaded it and called that agent tool and
then on your assistant you just have to have it that the user is asking a message on xyz document find the information along with some additional instructions for what you require. that should work.
id be also checking your api calls with debugs so you can see the raw inputs and outputs from the call to make sure there is not some error or context length issues where it error than the ai will just make something up depending on your logic to handle errors.