This browser is supported only in Windows 10 and above.

DID YOU READ

Our current star rating systems might deserve zero stars

Our current star rating systems might deserve zero stars (photo)

Posted by on

Those five star ratings systems on websites like Netflix and iTunes have turned us all into a nation of critics (sucks for me, I used to feel special). But is a five star ratings system really the best way to determine whether something is good or not? According to a couple of researchers from MIT, maybe not.

As spotted by the tech site GigaOM, a new paper by Devavrat Shah of MIT’s Laboratory of Information and Decisions Systems argues that online ratings systems “should instead ask users to compare products in pairs, not as stand-alone items.” In other words:

“…the kind of star rating systems that are the status quo on the web today are flawed because, well, humans are flawed. ‘If my mood is bad today, I might give four stars, but tomorrow I’d give five stars. But if you ask me to compare two movies, most likely I will remain true to that for a while,’ Shah says in an article published this week on MIT’s news site. ‘Your three stars might be my five stars, or vice versa. For that reason, I strongly believe that comparison is the right way to capture this.'”

Shah and his team believe that these “pairwise rankings” create a more accurate recommendation model than the five stars (they boast that their algorithm “accurately predict shoppers’ preferences with 20 percent greater accuracy than the kinds of formulas most often in use today”). In other other words, assigning a score to a single movie is a less reliable method of ranking films or television shows than selecting a preference between multiple films or television shows.

This is all tremendously nerdy stuff, so nerdy it’s giving my pocket protector sympathy pains. But it is pretty important, too. Any time you pay any attention to the star ratings on a website before you rent a movie on VOD, or purchase an album, or decide to make a reservation at a restaurant, you are validating those star ratings. And if they’re inaccurate, imagine the impact those inaccuracies have on your choices. If customers are being swayed by what those stars tell them, those stars damn well better be accurate.

Granted, I didn’t go to MIT and I can’t even spell algorithm much less make one of my own. I’m sure their research is sound and well thought-out. But my question would be this: how do they ensure people take pairwise rankings any more seriously than they do five star rankings? If web users, secure in their Internet anonymity, will give a movie one star just because they had a bad burrito for lunch, who’s to say they won’t react similarly when offered the choice of deciding between “Star Wars” and “Battlefield Earth?” How do we know they won’t use that same anarchic impulse to screw with those recommendations too? I’m not sure we do, but maybe there’s some sort of behavioral study that says that they won’t.

The other problem with comparative rankings is something I’ve found through my own casual use of the website Flickchart, where you repeatedly rank pairs of movies in order to create a list of your favorite movies. The site’s a lot of fun to play with, but the comparisons it proposes are often totally absurd. Comparing a thing you love to a thing you hate is easy: when Flickchart throws up “Almost Famous” and “Bad Boys 2” it’s not too hard to pick a winner. But comparing between things you love can very, very tricky.

If Flickchart gave you the comparison of movies above — “L.A. Confidential” and “Chinatown” — you’d have something to choose between. They’re both neo-noirs set in Los Angeles, they have similar tones and themes and ideas. It’s a hard pick, because they’re both phenomenal movies, but there are obvious commonalities. But if Flickchart asks you to compare between, “L.A. Confidential” and, say, “Snow White and the Seven Dwarfs” which do you select? Personally, I’m more likely to watch “L.A. Confidential” if given the choice between the two, but can I really argue “L.A. Confidential” is a “better” movie than “Snow White and the Seven Dwarfs?” “Snow White” is a watershed film in the history of cinema. “L.A. Confidential” is just a very well-acted and well-written crime story. So which one do I pick? And not only which one do I pick, but which one do other people pick? Some might choose according to their taste, some by a quasiobjective system of importance or cultural impact. The room for variation seems almost as large as with five star ratings.

I agree that five star ratings can be very flawed; there’s too much room for human error and not enough nuance. And I’m curious to hear and see more of this pairwise ranking in action. But if you gave me a pairwise ranking of these two options right now, I’m not sure what I would pick.

Your turn for some pairwise rankings: do you like five star ratings or comparative rankings? Tell us in the comments below or on Twitter and Facebook!

Watch More
Uncle-Buck

Give Back

Last-Minute Holiday Gift Guide

Hits from the '80s are on repeat all Christmas Eve and Day on IFC.

Posted by on
GIFs via Giphy, Photos via The Everett Collection

It’s the final countdown to Christmas and thanks to IFC’s movie marathon all Christmas Eve and Christmas Day, you can revel in classic ’80s films AND find inspiration for your last-minute gifts. Here are our recommendations, if you need a head start:

Musical Instrument

Great analog entertainment substitute when you refuse to give your kid the Nintendo Switch they’ve been drooling over.

Breakfast In Bed

Any significant other or child would appreciate these Uncle Buck-approved flapjacks. Just make sure you’re not stuck on clean up duty.

Cocktail Supplies

You’ll need them to get through the holidays.

Dance Lessons

So you can learn to shake-shake-shake (unless you know ghosts willing to lend a hand).

Comfy Clothes

With all the holiday meals, there may be some…embigenning.



Get even more great inspiration all Christmas Eve and Day on IFC, and remember…

Watch More
IFC_Portlandia-AORewind-blog

A-O Rewind

Celebrating Portlandia One Sketch at a Time

The final season of Portlandia approaches.

Posted by on
GIFs via Giphy

Most people measure time in minutes, hours, days, years…At IFC, we measure it in sketches. And nothing takes us way (waaaaaay) back like Portlandia sketches. Yes, there’s a Portlandia milepost from every season that changed the way we think, behave, and pickle things. In honor of Portlandia’s 8th and final season, Subaru presents a few of our favorites.

via GIPHY

Put A Bird On It

Portlandia enters the pop-culture lexicon and inspires us to put birds on literally everything.

Colin the Chicken

Who’s your chicken, really? Behold the emerging locavore trend captured perfectly to the nth degree.

Dream Of The ’90s

This treatise on Portland made it clear that “the dream” was alive and well.

No You Go

We Americans spend most of our lives in cars. Fortunately, there’s a Portlandia sketch for every automotive situation.

A-O River!

We learned all our outdoor survival skills from Kath and Dave.

One More Episode

The true birth of binge watching, pre-Netflix. And what you’ll do once Season 8 premieres.

Catch up on Portlandia’s best moments before the 8th season premieres January 18th on IFC.

Watch More
SistersWeekend_103_MPX-1920×1080

WTF Films

Artfully Off

Celebrity All-Star by Sisters Weekend is available now on IFC's Comedy Crib.

Posted by on

Sisters Weekend isn’t like other comedy groups. It’s filmmaking collaboration between besties Angelo Balassone, Michael Fails and Kat Tadesco, self-described lace-front addicts with great legs who write, direct, design and produce video sketches and cinematic shorts that are so surreally hilarious that they defy categorization. One such short film, Celebrity All-Star, is the newest addition to IFC’s Comedy Crib. Here’s what they had to say about it in a very personal email interview…

IFC_Comedy-Crib_Sisters-Weekend-Series-Image

IFC: How would you describe Celebrity All-Star to a fancy network executive you just met in an elevator?

Celebrity All-Star is a short film about an overworked reality TV coordinator struggling to save her one night off after the cast of C-List celebrities she wrangles gets locked out of their hotel rooms.

IFC: How would you describe Celebrity All-Star to a drunk friend of a friend you met in a bar?

Sisters Weekend: It’s this short we made for IFC where a talent coordinator named Karen babysits a bunch of weird c-list celebs who are stuck in a hotel bar. It’s everyone you hate from reality TV under one roof – and that roof leaks because it’s a 2-star hotel. There’s a magician, sexy cowboys, and a guy wearing a belt that sucks up his farts.

SistersWeekend_101_MPX-1920x1080

IFC: What was the genesis of Celebrity All-Star?

Celebrity All-Star was born from our love of embarrassing celebrities. We love a good c-lister in need of a paycheck! We were really interested in the canned politeness people give off when forced to mingle with strangers. The backstory we created is that the cast of this reality show called “Celebrity All-Star” is in the middle of a mandatory round of “get to know each other” drinks in the hotel bar when the room keys stop working. Shows like Celebrity Ghost Hunters and of course The Surreal Life were of inspo, but we thought it
was funny to keep it really vague what kind of show they’re on, and just focus on everyone’s diva antics after the cameras stop rolling.

IFC: Every celebrity in Celebrity All-Star seems familiar. What real-life pop personalities did you look to for inspiration?

Sisters Weekend: Anyone who is trying to plug their branded merch that no one asked for. We love low-rent celebrity. We did, however, directly reference Kylie Jenner’s turd-raison lip color for our fictional teen celebutante Gibby Kyle (played by Mary Houlihan).

IFC_Comedy-Crib_Sisters-Weekend_About-Image

IFC: Celebrity seems disgusting yet desirable. What’s your POV? Do you crave it, hate it, or both?

Sisters Weekend: A lot of people chase fame. If you’re practical, you’ll likely switch to chasing success and if you’re smart, you’ll hopefully switch to chasing happiness. But also, “We need money. We need hits. Hits bring money, money bring power, power bring fame, fame change the game,” Young Thug.

SistersWeekend_102_MPX-1920x1080

IFC: Who are your comedy idols?

Sisters Weekend: Mike grew up renting “Monty Python” tapes from the library and staying up late to watch 2000’s SNL, Kat was super into Andy Kaufman and “Kids In The Hall” in high school, and Angelo was heavily influenced by “Strangers With Candy” and Anna Faris in the Scary Movie franchise, so, our comedy heroes mesh from all over. But, also we idolize a lot of the people we work with in NY-  Lorelei Ramirez, Erin Markey, Mary Houlihan, who are all in the film, Amy Zimmer, Ana Fabrega, Patti Harrison, Sam Taggart. Geniuses! All of Em!

IFC: What’s your favorite moment from the film?

Sisters Weekend: I mean…seeing Mary Houlihan scream at an insane Pomeranian on an iPad is pretty great.

See Sisters Weekend right now on IFC’s Comedy Crib

Watch More