Addressing Performance and Value Discrepancies in OpenAI's GPT Models

Performance and Quality Concerns

While GPT-4o is positioned as a flagship model, our experiences and testing show that its performance in many areas does not consistently meet expectations. Specifically:

  • Quality of Responses: Gemini 2.0 Flash often produces responses that are equal to or greater in quality than GPT-4o and sometimes even surpassing GPT-4’s “o1” model. This is especially concerning given that GPT-4o is one of the flagship models of OpenAI, while Gemini 2.0 Flash is considered an “experimental” model.
  • “Dumbing Down” (Degradation): There have been noticeable instances where GPT-4o seems to be “dumbing down” or delivering less sophisticated responses than expected, suggesting a possible decrease in model quality over time.

Feature Discrepancies and Limitations

Despite being a paid service, GPT models also have notable limitations, particularly when compared to Gemini 2.0 Flash:

  • Multimodal Input: The “o1” model does not support file and video inputs, while Gemini 2.0 Flash handles these inputs without issues. Although GPT-4o has multimodal capabilities, they are not yet fully implemented across all platforms. Specifically, while it can handle text, images and audio, video input requires a workaround of processing frames as images.
  • Video Input: The latest iteration of the GPT models, GPT-4o, doesn’t accept videos as direct input. To analyze a video, users need to extract frames and input them as a series of images, which is cumbersome. GPT-4o does have the capacity to use the frames to describe the video, and even generate a voiceover using the TTS API.

Value and Pricing Concerns

The cost of using GPT-4o and “o1” is also a significant concern, particularly considering the performance issues:

  • Pricing: Both GPT-4o and “o1” are priced at $20 per month, whereas Gemini 2.0 Flash is currently free.
  • Value Proposition: The combination of lower quality, limitations in features, and high costs makes the value proposition of OpenAI’s models less compelling compared to Gemini 2.0 Flash.

Lack of Competitive Advantage

Given these concerns, we are worried about OpenAI’s current standing in the AI field. The lack of superior performance or exclusive features does not provide a competitive advantage. The fact that a free model (Gemini 2.0 Flash) often delivers better results raises questions about the sustainability of OpenAI’s current model and pricing strategies.

Summary of Issues

Here’s a concise list of the problems mentioned above:

  1. Response Quality: Gemini 2.0 Flash often matches or surpasses the quality of GPT-4o and sometimes “o1” responses.
  2. Model Degradation: Perceived “dumbing down” or reduced sophistication in GPT-4o responses.
  3. Limited Multimodal Input: “o1” model lacks file and video input support, while Gemini 2.0 Flash has it. Although, GPT-4o is multimodal, its video input requires a workaround.
  4. Video Input: GPT-4o doesn’t support direct video input, requiring the use of video frames as images.
  5. High Cost: GPT-4o and “o1” are priced at $20 per month, while Gemini 2.0 Flash is free.
  6. Value Imbalance: The high cost does not align with the perceived quality and feature set compared to Gemini 2.0 Flash.
  7. Competitive Disadvantage: Lack of clear advantages over free competitors in performance, features, and pricing.

We urge OpenAI to investigate these issues and prioritize the following:

  • Improve Model Quality: Focus on refining models to consistently deliver high-quality, sophisticated responses.
  • Enhance Multimodal Capabilities: Expand the range of supported input types, especially full, direct video input, without the need for workarounds.
  • Re-evaluate Pricing: Adjust pricing to better reflect the performance and features offered, particularly in comparison to competitive alternatives.
  • Address User Feedback: Take user feedback on model quality seriously and take proactive steps to improve the user experience.

We believe that addressing these issues is crucial for OpenAI to maintain its position in the AI market and provide users with the high-quality tools that they have come to expect.