Valve says it is refining its attitude to how customer reviews are handled on Steam, and today announces that it plans to identify and remove "off-topic review bombs" that can affect the user review result.
Review bombing has been a long-standing problem for digital user-rated stores, and Valve has been trying to combat their impact on Steam with other methods, including reviews of user reviews and charts showing "temporary distortions" in user feedback. But so far, Valve has still introduced these reviews into the game's total user value.
And while some review cases may be related to legitimate gaming problems, many are unrelated or tangentially related to the product itself. Games like Firewatch PlayerUnknown's Battlegrounds and Dota 2 have been directed against waves of negative reviews for incidents far from the scope of the games themselves, either over political and ideological disagreements or decisions made in other games by the developers. Horror Games Devotion is the latest high-profile example of this behavior; The game's steam page was flooded with negative reviews after users discovered a game console that was behind China's president.
Everyone leads to an obvious question: How does Valve consider something "off-topic"? Valve is explained in a post on Steam how it determines it and the mix of automated tools and human interaction that will help confirm it.
We define a survey bomb as a focus where we have a topic that we do not consider to be related to the likelihood that future buyers will be happy if they buy the game, and thus not something to be added to the audit result.
It is obvious that there is a gray area here because there is a wide variety of things that players care about. So how do we identify these survey bombs? The first step is a tool we have built that identifies any divergent review activity on all games on Steam in as near real-time as possible. It does not know why a particular game receives aberrant review activity, and it does not even try to figure it out. Instead, it announces a team of people at Valve, who then goes and investigates. We've already run our tool across the full story of Steam reviews, and identify many reasons why games have seen periods of divergent review activity, and bombs that are not subject to investigation seem to be just a few of them.
Valve said that some substances that are closely related to games, especially DRM (Digital Rights Management) programs and End User License Agreements (EULA), will fall within the subject area.
We had long debates about these two and others like them. They are technically not part of the game, but they are a problem for some players. In the end, we have decided to define them as bomb investigations. Our reasoning is that the "general" repeater does not care so much about them, so the audit result is more accurate if it does not contain them. In addition, we believe that players who care about topics such as DRM are often willing to dig a little deeper into games before they buy ̵1; that's why we still hold all reviews within the review bombs. It only takes a minute to dig into those reviews to see if the problem is something you care about.
And here's what gets harder: Valve's user user identification reviews will also remove neutral and positive reviews from the overall score calculation if they were made during the bombing window. Valve says "our data shows that review bombs tend to be temporary distortions, so we believe the review score is still accurate and that other players can still find and read your review within the period."
More details on the change are available on Steam.