OpenAI Evals - plans for future?

[dmitry]

Hey,
so I’ve been busy last week on researching how to build a custom eval tool and OpenAI Evals gave me a great start. It is, however, quite basic and unfortunately not very extendable. I’d love to contribute (not evals, but the core code itself), but not sure what are the plans for the future of the repo, and if it will be actively maintained. I wasn’t able to find any OpenAI Evals communities. Would appreciate if anyone can shed some light.

Thank you!

[a6573623]

@dmitry

so I’ve been busy last week on researching how to build a custom eval tool and OpenAI Evals gave me a great start. It is, however, quite basic and unfortunately not very extendable. I’d love to contribute (not evals, but the core code itself), but not sure what are the plans for the future of the repo, and if it will be actively maintained. I wasn’t able to find any OpenAI Evals communities. Would appreciate if anyone can shed some light.

Hey, I am in your shoes right now. Wondering if you found any answers?

[dmitry]

Hey, sorry for late reply. So far I didn’t see any progress with open ai evals repo. I decided to move forward with Llama Index evaluations. It’s a pretty fresh module, but library develops very quickly and has great team behind it. Here are their docs: Evaluation - LlamaIndex 🦙 0.8.43