Review Aggregators: Certified Fresh or Blowing off Steam?

As the saying goes: everyone’s a critic.

“The Promise,” a soon-to-be-released movie set in the last days of the Ottoman Empire, has garnered over 80,000 user reviews on the Internet Movie Database (IMDb). Yet, it has not yet been released in the United States, and only premiered at a Canadian film festival on September 11.

Sixty-four percent of the ratings give the film one star out of ten, driving the score down. Some speculate the ratings are a result of the movie addressing the Armenian genocide, even though professional critics have had mostly positive receptions to the film. Despite the poor general ratings, 35 percent of raters give the film a perfect score.

Manipulated reviews, which some audiences claim plague online review aggregators, are not exclusive to movies. On Steam, an online video game distribution platform, the early access game “Otherland” was found to have fake reviews on the game’s page: reviewers left a variety of comments on the game, despite having zero hours of gameplay on record.

But even when an online review score is only based on professional critic’s opinions, it is rare that everyone is satisfied. Online movie review sites like Rotten Tomatoes and Metacritic, which act as aggregators for critics’ “professional” opinions, have frequently been slammed with petitions from viewers of certain movies that disagree with the critics’ response.

One petition to shut down the movie review site Rotten Tomatoes was inspired by the low critics’ rating of 26 percent for the movie “Suicide Squad”, which racked up an audience score of a whopping 67 percent. The petition has since closed, but its intent was made it clear in its description. “There’s a disconnect between critics and audiences,” the petition read, “you may enjoy a movie regardless what the critics say about it. We must get the people to know that the criticism not the measure of the quality of movies, it’s just the opinions of the critics.”

For the weekend of September 9th to 12th, three out of the five top grossing movies were rotten (indicating particularly low scores from critics), according to the site.

The disconnect begs the question, why do all the critics’ opinions need to be brought together and averaged?

In May, fans of the final installment in the Nathan Drake video game series, “Uncharted 4: A Thief’s End,” found themselves asking the same question when a Washington Post review rattled a nearly perfect score on Metacritic.

Their answer? Petition to have the review removed.

Out of 112 reviews on Metacritic for the game, 110 were positive, one was mixed, leaving only one negative review.

The review from the Washington Post did not include a numerical rating, but on Metacritic, the review was accompanied by a bright red numeral: 40 percent.

On the petition, signers claimed the review wasn’t comprehensive. “This is a disgraceful review and shouldn’t have been counted. It’s pure idiocy and an awful rating with nothing to back it up,” said Stephane Barbeau, a signer on the petition. “(The Washington Post reviewer) gave a 4/10 simply cause he didn’t like the story and never stated anything about the other aspects of the game.”

But is there any way to moderate what reviews should and should not be counted before users have to turn to petitions?

Steam unveiled a new review policy in an attempt to avoid inflated or out-of-date reviews. Now, the default calculation of the review that users will initially see at the top of game pages filters out reviews from users who obtained the game through Steam keys, which many users receive to download games for free. 

Many say this is an attempt to avoid what happened with “Otherland,” but independent game developers are upset it could hurt their review scores, and thus, the success of their game as they struggle against established titles from bigger studios. The smaller studios will often give out these keys to backers on Kickstarter, or sell the game through their own site.

Simon Roth, developer for the game Maia, claimed that these changes were him going out of business as scores were removed from the game’s score.

Other developers, though, are positive about the restrictions. David Pittman is confident in his own game and the people who bought it. With the change in the system, reviews for his game actually went up, according to his Twitter.

Even moderating reviews is leaving people split. There seems to be no way to make everyone happy with any system for the reviews. Steam’s new policy could open doors for unbiased reviews, but time will tell for its effectiveness—and if others will get on board.

Leave a Reply

Your email address will not be published.