In your opinion, are LLM companies chasing benchmarks too much?

What do you guys think?

it certainly feels this way recently, where new LLM models are boasted on social media to improve on benchmarks, but in the “real-world-usage” it feels like its regressing.

All community opinions are welcomed! Maybe you guys have a different perspective and different use cases!

(this topic follows all community guidelines)

3 Likes