This browser is supported only in Windows 10 and above.

DID YOU READ

Our current star rating systems might deserve zero stars

Our current star rating systems might deserve zero stars (photo)

Posted by on

Those five star ratings systems on websites like Netflix and iTunes have turned us all into a nation of critics (sucks for me, I used to feel special). But is a five star ratings system really the best way to determine whether something is good or not? According to a couple of researchers from MIT, maybe not.

As spotted by the tech site GigaOM, a new paper by Devavrat Shah of MIT’s Laboratory of Information and Decisions Systems argues that online ratings systems “should instead ask users to compare products in pairs, not as stand-alone items.” In other words:

“…the kind of star rating systems that are the status quo on the web today are flawed because, well, humans are flawed. ‘If my mood is bad today, I might give four stars, but tomorrow I’d give five stars. But if you ask me to compare two movies, most likely I will remain true to that for a while,’ Shah says in an article published this week on MIT’s news site. ‘Your three stars might be my five stars, or vice versa. For that reason, I strongly believe that comparison is the right way to capture this.'”

Shah and his team believe that these “pairwise rankings” create a more accurate recommendation model than the five stars (they boast that their algorithm “accurately predict shoppers’ preferences with 20 percent greater accuracy than the kinds of formulas most often in use today”). In other other words, assigning a score to a single movie is a less reliable method of ranking films or television shows than selecting a preference between multiple films or television shows.

This is all tremendously nerdy stuff, so nerdy it’s giving my pocket protector sympathy pains. But it is pretty important, too. Any time you pay any attention to the star ratings on a website before you rent a movie on VOD, or purchase an album, or decide to make a reservation at a restaurant, you are validating those star ratings. And if they’re inaccurate, imagine the impact those inaccuracies have on your choices. If customers are being swayed by what those stars tell them, those stars damn well better be accurate.

Granted, I didn’t go to MIT and I can’t even spell algorithm much less make one of my own. I’m sure their research is sound and well thought-out. But my question would be this: how do they ensure people take pairwise rankings any more seriously than they do five star rankings? If web users, secure in their Internet anonymity, will give a movie one star just because they had a bad burrito for lunch, who’s to say they won’t react similarly when offered the choice of deciding between “Star Wars” and “Battlefield Earth?” How do we know they won’t use that same anarchic impulse to screw with those recommendations too? I’m not sure we do, but maybe there’s some sort of behavioral study that says that they won’t.

The other problem with comparative rankings is something I’ve found through my own casual use of the website Flickchart, where you repeatedly rank pairs of movies in order to create a list of your favorite movies. The site’s a lot of fun to play with, but the comparisons it proposes are often totally absurd. Comparing a thing you love to a thing you hate is easy: when Flickchart throws up “Almost Famous” and “Bad Boys 2” it’s not too hard to pick a winner. But comparing between things you love can very, very tricky.

If Flickchart gave you the comparison of movies above — “L.A. Confidential” and “Chinatown” — you’d have something to choose between. They’re both neo-noirs set in Los Angeles, they have similar tones and themes and ideas. It’s a hard pick, because they’re both phenomenal movies, but there are obvious commonalities. But if Flickchart asks you to compare between, “L.A. Confidential” and, say, “Snow White and the Seven Dwarfs” which do you select? Personally, I’m more likely to watch “L.A. Confidential” if given the choice between the two, but can I really argue “L.A. Confidential” is a “better” movie than “Snow White and the Seven Dwarfs?” “Snow White” is a watershed film in the history of cinema. “L.A. Confidential” is just a very well-acted and well-written crime story. So which one do I pick? And not only which one do I pick, but which one do other people pick? Some might choose according to their taste, some by a quasiobjective system of importance or cultural impact. The room for variation seems almost as large as with five star ratings.

I agree that five star ratings can be very flawed; there’s too much room for human error and not enough nuance. And I’m curious to hear and see more of this pairwise ranking in action. But if you gave me a pairwise ranking of these two options right now, I’m not sure what I would pick.

Your turn for some pairwise rankings: do you like five star ratings or comparative rankings? Tell us in the comments below or on Twitter and Facebook!

IFC_Portlandia-S8_best-of-skits_subaru-blog

Final Countdown

The Best Of The Last

Portlandia Goes Out With A Bang

Posted by on

The end is near. In mere days Portlandia wraps up its final season, and oh what a season it’s been. Lucky for you, you can watch the entire season right now right here and on the IFC app, including this free episode courtesy of Subaru.

But now, let’s take a moment to look back at some of the new classics Fred and Carrie have so thoughtfully bestowed upon us. (We’ll be looking back through tear-blurred eyes, but you do you.)

Couples Dinner

It’s not that being single sucks, it’s that you suck if you’re single.

Cancel it!

A sketch for anyone who has cancelled more appointments than they’ve kept. Which is everyone.

Forgotten America

This one’s a “Serial” killer…everything both right and wrong about true crime podcasts.

Wedding Planners

The only bad wedding is a boring wedding.

Disaster Hut

It’s only the end of the world if your doomsday kit doesn’t include rosé.

Catch up on Portlandia’s final episodes on demand and at IFC.com

IFC_Portlandia-S8_pick-a-lane_subaru-blog

Rev Up

Your Portlandia Personality Test

The New Portlandia Webseries Is Going Your Way

Posted by on

Carrie and Fred understand that although we have so much in common, we’re each so beautifully unique and different. To help us navigate those differences, Portlandia has found an easy and honest way to embrace our special selves in the form of a progressive new traffic system: a specific lane for every kind of driver. It’s all in honor of the show’s 8th and final season, and it’s all presented by Subaru.

Ready to find out who you really are? Match your personality to a lane and hop on the expressway to self-understanding.

Lane 10: Trucks Piled With Junk

Your junk is falling out of your trunk. Shake a tail light, people — this lane is for you.

Lane 33: Twins

You’re like a Gemini, but waaaay more pedestrian. Maybe you and a friend just wear the same outfits a lot. Who cares, it’s just twinning enough to make you feel special.

Lane 27: Broken Windows

Bad luck follows you around and everyone knows it. Your proverbial seat is always damp from proverbial rain. Is this the universe telling you to swallow your pride? Yes.

Lane 69: Filthy Cars

You’re all about convenience. Getting your car washed while you drive is a no-brainer.

Lane 43: Newly Divorced Singles

It’s been a while since you’ve driven alone, and you don’t know the rules of the road anymore. What’s too fast? What’s too slow? Are you sending the right signals? Don’t worry, the breakdown lane is nearby if you need it.

Still can’t find a lane to match your personality? Check out all the videos here. And see the final season of Portlandia this spring on IFC.

Uncle-Buck

Give Back

Last-Minute Holiday Gift Guide

Hits from the '80s are on repeat all Christmas Eve and Day on IFC.

Posted by on
GIFs via Giphy, Photos via The Everett Collection

It’s the final countdown to Christmas and thanks to IFC’s movie marathon all Christmas Eve and Christmas Day, you can revel in classic ’80s films AND find inspiration for your last-minute gifts. Here are our recommendations, if you need a head start:

Musical Instrument

Great analog entertainment substitute when you refuse to give your kid the Nintendo Switch they’ve been drooling over.

Breakfast In Bed

Any significant other or child would appreciate these Uncle Buck-approved flapjacks. Just make sure you’re not stuck on clean up duty.

Cocktail Supplies

You’ll need them to get through the holidays.

Dance Lessons

So you can learn to shake-shake-shake (unless you know ghosts willing to lend a hand).

Comfy Clothes

With all the holiday meals, there may be some…embigenning.



Get even more great inspiration all Christmas Eve and Day on IFC, and remember…