Creating Evals that accept file inputs

I have a request that provide file inputs and text. E.g. a base64 file is provided along with a request “please summarize the file”.

How can I build evals for this type of prompt that includes files and text?

I’ve tried setting the test data with base64 encoded file data to feed as input into a jsonl file. When calling the eval run, i declare input messages similar to how I would when making a request to responses.

const run = await openai.evals.runs.create("eval-...", {
    name: "Categorization summary",
    data_source: {
        type: "responses",
        model: "gpt-4.1",
        input_messages: {
            type: "template",
            template: [
                { role: "developer", content: "prompt...." },
                {
                    role: "user",
                    content: [
                        {
                            type: "input_file",
                            filename: "draconomicon.pdf",
                            file_data: "{{item.file_content}}"
                        },
                        {
                            type: "input_text",
                            text: "What is the first dragon in the book?",
                        },
                    ],
                },
            ],
        },
        source: { type: "file_id", id: "file-..." },
    },
});