Evals not running and facing repeated errors

API Bugs
1

I am not able to run the evals script. My yaml file is not recognized from registry and getting error like below even after trying multiple times:

The image shows a terminal window displaying a Python traceback error, specifically an AssertionError indicating that 'simple_eval/is_evaluation_good.dev' is not found in the list of available evaluations. (Captioned by AI)
The image shows a terminal window displaying a Python traceback error, specifically an AssertionError indicating that 'simple_eval/is_evaluation_good.dev' is not found in the list of available evaluations. (Captioned by AI)1640×315 27.8 KB

2

Hi Rajalakshmi :waving_hand: Welcome to the community!

The error you’re seeing means the custom eval ID you’re trying to run (simple_eval/is_evaluation_good.dev) is not registered in evals/registry/evals YAML files.

To get started with building and running evals, you can follow this official guide:
:link: Getting Started with OpenAI Evals

1 Like