We’ve got one week left in 2011, so we’re running out of time to make lists, tear apart other people’s lists, make lists of lists, and list the lists we are going to list later. Better make the most of it.
Last night, for example, I spent a couple of hours pouring over the two big annual film critic polls: one from The Village Voice, the other from Indiewire. In both cases, the publications invite dozens of critics to list their favorite films, performances, directors, and assorted other topics. They compile the responses and use them to generate rankings. There’s a pretty large overlap between the two polls — out of the 193 total participants, 63 critics submitted a ballot to both — and some interesting disparities between the two sets of results.
Both polls agree about the best movie of the year: Terrence Malick‘s “The Tree of Life,” which appeared on more than half the Voice ballots and over a third of the Indiewire ballots. But after that, the lists get jumbled — while the two polls share nine out of the ten same films, no other movies occupies the same position on both top ten lists. The #2 and #3 films, “A Separation” and “Melancholia,” are flipped depending on which poll you look at — thorny matters of Iranian divorce were more popular in the Voice while greeting the end of all existing with Danish indifference was more popular with the Indiewire voters. The single biggest difference in placements for one film in the two top ten lists was “Drive,” which ranked as the fifth most popular film of the year on Indiewire but just the ninth most popular in the Voice. The outliers that only appeared on one poll each were “Hugo” (eighth on Indiewire, eleventh in the Voice) and “Take Shelter” (tenth in the Voice poll, fourteenth on Indiewire).
There are similar discrepancies in the acting categories, with a lot of repeated selections in vastly different orders of preference. Picking the best lead performance of the year, Indiewire voters wound up with a tie between Michael Shannon in “Take Shelter” and Michael Fassbender in “Shame.” But in the Voice, Shannon was the runaway favorite, receiving almost twenty more points than anyone else in either gender, while Fassbender came in fifth place behind Anna Paquin from “Margaret,” Juliette Binoche from “Certified Copy,” and Kirsten Dunst from “Melancholia.” Fassbender appeared on 27% of all Indiewire ballots and just 21% of all Voice ballots, a pretty big difference especially when you consider that one out of every three ballots in both polls were essentially identical. On the supporting side of things, Christopher Plummer got similarly Fassbendered. At Indiewire he won Best Supporting Performance by a sizable margin. In the Voice he only placed third, behind Albert Brooks in “Drive” and Jeannie Berlin in “Margaret.”
So what does this all mean? What do I look like, a guy who took more than one math class in college? Because I didn’t. Personally, I think it means that while consensus does exist out there in film critic land, it’s also far more fickle and flexible than we often imagine it to be. You poll 10 critics, you might get complete agreement. You poll 10 other critics, you might get ten different favorites. A few critics invited to vote here, a few critics not invited to vote there, and intentionally or unintentionally you’ve created significant variations in the data.
If you were going to take this research even further down sabremetriciany avenues, you’d need information that the Voice and Indiewire don’t publicly provide, namely the ages and outlets of their contributors. Then you could compare the statistical variations in the two polls with other factors; maybe the average voting age at Indiewire was younger and younger voters tended to prefer “Drive.” Or maybe print critics were more heavily sampled at the Voice, and they were less impressed by Michael Fassbender’s emotionally naked
For now, I guess we’re left more questions. In the meantime, be sure to examine the Voice and Indiewire polls in depth and to report back to me with your own findings. Or if you just want to read my ballots you can find those here and here.