Would you like an AB test framework for LangChain projects?

Prompting
, ,
1

While creating my startup product, I had a tough time seeing how changes to prompts made a difference.

So, I’m considering starting an open-source project for A/B testing AI prompts. It’s all about making it easier to handle prompts and few-shot examples, run tests, get feedback, and see results.

But before I dive in, I’d love to know if this tool could help others too. Does it sound handy to you?

3 Likes
2

I think A/B testing is a great idea for prompts.

Considering it’s open source maybe you could frame it in such a way that it can be easily modified to work with Langchain?

2 Likes
3

Thank you for your positive feedback @RonaldGRuckus :slight_smile: Yes, right! I’m considering a web application that works with LangChain project.