Has anyone else found that their GPTs are worse at following instructions when you provide files as context?

I’m creating a content creator custom GPT for our team at work. The instructions in the configuration are fairly specific with step by step interactions (e.g. ask in this order, output in this way only etc.).

For the GPT to create the content, it needs source material. I’ve tested it in two ways:

  1. by providing the source material as an attached PDF
  2. by directly pasting in the text.

Now, I expected method 1 to lead to poorer content retrieval, but what I didn’t expect is for the GPT to become ‘dumber’ in general and less accurate in following my instructions. It is far less reliable in following the configuration parameters (e.g asking questions in order, outputting in a certain format, etc.) compared to method 2, where I don’t attach a PDF as context.

Has anyone else noticed this? Is this a bug or just an expected tradeoff with working with PDFs as context?