The Problem With Relying on Aggregators

Posted by Alison Willmore on September 13th

In the comments for Pete Hammond’s report on Disney’s Oscar push for the upcoming “Tangled,” a chorus of voices chimes their support for the greatness of “How to Train Your Dragon.” And last week, the Hollywood Reporter‘s Borys Kit noted that DreamWorks Animation had sent out a best animated feature and best picture For Your Consideration packet for the film: “The fact that it’s supporting the movie isn’t the unusual part — the movie is easily one the best films of the year so far — but it’s not even Labor Day yet!”

It is? Easily? As bemusing as I find this glimpse into an alternate universe of film valuation, it’s nothing new — the more films there are, the more they target very specific audiences, the higher the chances that everyone will have (very) different ideas about the what the best one of the year will be (and the argument over what constitutes quality is an eternal one). But I do take issue with this, from Kit’s piece:

“Dragon” has an awesome 98% rating on review aggregator site Rotten Tomatoes so you’d think it would have a great edge. But look out! Its rival, “Toy Story 3” from practically perennial winner Pixar, has 99%.

Rotten Tomatoes measures consensus, not quality. A 98% or 99% rating on the site means that almost everyone agreed that it was good — but that doesn’t mean that more consensus is proof of a better movie. Great movies — challenging, ambitious, weighty, unusual — tend to be more divisive, to invite high praise and deep disagreement. An innocuous movie that everyone thinks is absolutely adequate could obtain a higher RT rating than one that the majority of critics claim is the best of the decade, and a few others can’t stand. That doesn’t mean that the former is better than the latter.

This is what the angry types that attack whoever dares spoil a favorite movie’s “100% Fresh” rating (usually Armond White) miss — that everyone agreeing that a film is good is worth a lot less than writers making interesting, well-reasoned cases for why that film is great. Rotten Tomatoes and other aggregators are very useful for getting a glimpse of the zeitgeist critical opinion of a movie and for finding in depth reviews, but to place so much significance on percentage points is ridiculous. A 98% approval rating isn’t proof that a film is the best of the year, or proof that it isn’t. It’s proof that most people found it perfectly watchable.