I often ask GPT to research people, products, games, and other things I am unfamiliar with. This is one of the features I find most useful. These days, GPT can often “Google” something much faster and more thoroughly than I can on my own.
However, lately I have been increasingly dissatisfied with the quality of some of its research results, especially when I ask it to investigate something that is not widely known yet.
This seems to happen particularly often with obscure or relatively unknown games.
For example, there may be a small game with overwhelmingly positive user reviews, along with multiple external reviews that discuss the game in detail and evaluate it positively. Yet GPT sometimes ignores most of that evidence and instead focuses heavily on a single negative blog review.
The problem is not simply that the review is negative. Negative reviews can obviously be useful and should absolutely be included in research.
The problem is that the review GPT chooses to emphasize may itself be extremely low quality. It may contain very little meaningful analysis of the game, barely discuss its mechanics or content, and consist mostly of shallow criticism or deliberately harsh commentary. Despite that, GPT may treat this one review as if it were especially important or representative when forming its overall assessment.
If I then ask GPT to research the same game again, but this time to examine it in much greater detail, the result can change significantly. Once it looks at more detailed reviews, broader user feedback, community discussions, and analyses written by people who actually explain the game’s systems and their experience with it, the overall picture may be completely different from what that one negative reviewer claimed.
That is the part I find concerning.
Again, my issue is not that GPT includes criticism or negative opinions. A good research process should include both positive and negative perspectives.
The issue is that GPT sometimes seems to give disproportionate weight to negative opinions without adequately considering the quality of the source, the depth of the analysis, the amount of actual experience behind the review, or how representative that opinion is compared with the broader reception.
At times, it feels as though GPT assumes that because I am a potential consumer, the safest or most helpful approach is to prioritize negative reviews and warnings.
But I do not think that necessarily serves consumers well.
If GPT gives too much weight to an unverified or low-quality negative review simply because it is negative, it can actually mislead a potential customer who might otherwise be a very good fit for that game or product.
In other words, an attempt to be “consumer-friendly” can end up doing the opposite. It can discourage people from products they might genuinely enjoy because a low-quality negative opinion was given more weight than a much larger body of detailed and positive feedback.
I would therefore like to see improvements in how GPT evaluates and weighs sources during search and research.
Rather than treating reviews primarily according to whether they are positive or negative, GPT should also consider how specific and well-supported each review is, how much direct experience the reviewer appears to have with the subject, and how deeply the reviewer actually analyzes it.
And I think this evaluation should go beyond the individual review itself. GPT should also investigate the reviewer’s broader track record. What is the usual quality of that person’s reviews? Do they generally provide detailed, well-reasoned analysis, or do they have a history of producing shallow, poorly supported, or deliberately provocative reviews? How consistent is their methodology across different games or products? What kind of work or activity does the reviewer engage in outside that particular review, and what is their broader reputation among readers, other reviewers, or relevant communities?
None of these factors should automatically invalidate a review, but they are important context when deciding how much weight that review deserves. If GPT is going to rely heavily on a particular critic or blogger when evaluating an obscure product, it should investigate the credibility and track record of that source at the same time instead of treating every published opinion as equally reliable.
It would also be helpful if GPT paid more attention to the overall distribution of user feedback and compared multiple independent sources instead of allowing one strongly worded minority opinion to dominate the final assessment.
This is especially important for obscure games, products, or other subjects where relatively little information is available. In those cases, GPT should be particularly careful not to over-rely on one or two highly ranked search results simply because they happen to appear prominently.
I also think GPT should distinguish between criticism that is detailed and well-supported and criticism that is merely aggressive, provocative, or strongly worded. A harsh opinion should not automatically be treated as more informative or more credible simply because it sounds more skeptical.
Finally, I think GPT would often be more useful if it avoided reducing its conclusion to “this product is good” or “this product is bad.” Instead, it should explain which kinds of users are likely to appreciate certain aspects of the product, which kinds of users may dislike them, and how those differences relate to the broader reception.
GPT’s search and research capabilities are already extremely useful. But I think they could become much more reliable if the system improved not only how much information it finds, but also how it evaluates source quality, reviewer credibility, representativeness, and the relative weight that different opinions deserve.