Hi there
I’m following the guide “Moving from OpenAI Evals to Promptfoo” to move from evals to promptfoo. It mentions a button to export as a runnable promptfoo config.
I cannot find this in the real interface.
Is that feature not implemented yet, or has anyone else had success migrating from OpenAI Evals to promptfoo?
Hi and welcome to the community @cboisen!
I haven’t tried migrating Evals to Promptfoo myself yet, but I found the section in the OpenAI cookbook article that I think you’re referring to.
The steps are:
The article page also has View on GitHub / Download raw.
Maybe you already know and have tried those steps.
If any of those aren’t to any help for you, hopefully someone else has already migrated and can give you a right guidance here.
That is the cookbook I was referring to, yes. I was not allowed to make a link in my post.
These steps refer to a menu that doesn’t exist when I open the interface. Step 3 to be exact.
And what happens when you click export? I get a jsonl file download, not a runnable promptfoo configuration, right?
I’m sorry if I come off as annoyed or arrogant, but I must admit I feel you are wasting time by only suggesting to proceed “one step further” than your previous message, and not actually walking through the entire flow.
Suggesting I need to have promptfoo installed isn’t helpful, nor relevant to the issue I’m pointing out.
I am sorry, I totally misunderstood what you meant in your previous post. I thought you meant the Export menu itself was not visible, that’s on me.
Hi @cboisen, this one's definitely confusing. The key point is that the migration guide is referring to two different export types.
A runnable Promptfoo configuration includes the prompts, models, test cases, assertions, and other configuration needed to rerun the evaluation. That's what the guide means by exporting "supported evaluations."
A JSONL export is different. It only contains the results of a completed evaluation, so it's useful for comparing historical runs, but it doesn't include enough information to recreate a runnable Promptfoo config.
If you're only seeing JSONL, you're most likely exporting a completed run rather than the evaluation definition. Also, not every evaluation type currently supports exporting as a Promptfoo configuration.
Can you share whether you're using the newer OpenAI Platform Evals or the older openai/evals GitHub framework? Also, which page are you exporting from: the evaluation details page or a completed run?
-Mark G.
