I’m curious if anyone has tested the new GPT-4o model using the “Needle in a Haystack” benchmark. If not, perhaps the developers could share their insights on how it might perform in comparison to the GPT-4 Turbo. Any feedback or data would be greatly appreciated!

Performance of GPT-4o on the Needle in a Haystack Benchmark

Jirito May 15, 2024, 4:35pm 4

This is what I was looking for, thank you!

At the end of the day, when it comes to needle in a haystack, Claude 3 Opus still outperforms gpt4o, no?

2 Likes

Topic		Replies	Views
GPT4o - performance comparison with GPT4 API gpt-4	0	598	June 7, 2024
GPT-4-Turbo models perform better the older GPT-4 models in LMSys benchmark API gpt-4 , api	14	6317	May 13, 2024
Worse results when using GPT-4o as an evaluator Community gpt-4o , evaluation	2	267	October 1, 2024
Did anyone try new gpt4 o model for text extraction from an image? API gpt-4 , chatgpt	2	1464	June 10, 2024
GPT-4o vs. gpt-4-turbo-2024-04-09, gpt-4o loses API gpt-4	38	14015	June 11, 2024