This browser is supported only in Windows 10 and above.

DID YOU READ

Our current star rating systems might deserve zero stars

Our current star rating systems might deserve zero stars (photo)

Posted by on

Those five star ratings systems on websites like Netflix and iTunes have turned us all into a nation of critics (sucks for me, I used to feel special). But is a five star ratings system really the best way to determine whether something is good or not? According to a couple of researchers from MIT, maybe not.

As spotted by the tech site GigaOM, a new paper by Devavrat Shah of MIT’s Laboratory of Information and Decisions Systems argues that online ratings systems “should instead ask users to compare products in pairs, not as stand-alone items.” In other words:

“…the kind of star rating systems that are the status quo on the web today are flawed because, well, humans are flawed. ‘If my mood is bad today, I might give four stars, but tomorrow I’d give five stars. But if you ask me to compare two movies, most likely I will remain true to that for a while,’ Shah says in an article published this week on MIT’s news site. ‘Your three stars might be my five stars, or vice versa. For that reason, I strongly believe that comparison is the right way to capture this.'”

Shah and his team believe that these “pairwise rankings” create a more accurate recommendation model than the five stars (they boast that their algorithm “accurately predict shoppers’ preferences with 20 percent greater accuracy than the kinds of formulas most often in use today”). In other other words, assigning a score to a single movie is a less reliable method of ranking films or television shows than selecting a preference between multiple films or television shows.

This is all tremendously nerdy stuff, so nerdy it’s giving my pocket protector sympathy pains. But it is pretty important, too. Any time you pay any attention to the star ratings on a website before you rent a movie on VOD, or purchase an album, or decide to make a reservation at a restaurant, you are validating those star ratings. And if they’re inaccurate, imagine the impact those inaccuracies have on your choices. If customers are being swayed by what those stars tell them, those stars damn well better be accurate.

Granted, I didn’t go to MIT and I can’t even spell algorithm much less make one of my own. I’m sure their research is sound and well thought-out. But my question would be this: how do they ensure people take pairwise rankings any more seriously than they do five star rankings? If web users, secure in their Internet anonymity, will give a movie one star just because they had a bad burrito for lunch, who’s to say they won’t react similarly when offered the choice of deciding between “Star Wars” and “Battlefield Earth?” How do we know they won’t use that same anarchic impulse to screw with those recommendations too? I’m not sure we do, but maybe there’s some sort of behavioral study that says that they won’t.

The other problem with comparative rankings is something I’ve found through my own casual use of the website Flickchart, where you repeatedly rank pairs of movies in order to create a list of your favorite movies. The site’s a lot of fun to play with, but the comparisons it proposes are often totally absurd. Comparing a thing you love to a thing you hate is easy: when Flickchart throws up “Almost Famous” and “Bad Boys 2” it’s not too hard to pick a winner. But comparing between things you love can very, very tricky.

If Flickchart gave you the comparison of movies above — “L.A. Confidential” and “Chinatown” — you’d have something to choose between. They’re both neo-noirs set in Los Angeles, they have similar tones and themes and ideas. It’s a hard pick, because they’re both phenomenal movies, but there are obvious commonalities. But if Flickchart asks you to compare between, “L.A. Confidential” and, say, “Snow White and the Seven Dwarfs” which do you select? Personally, I’m more likely to watch “L.A. Confidential” if given the choice between the two, but can I really argue “L.A. Confidential” is a “better” movie than “Snow White and the Seven Dwarfs?” “Snow White” is a watershed film in the history of cinema. “L.A. Confidential” is just a very well-acted and well-written crime story. So which one do I pick? And not only which one do I pick, but which one do other people pick? Some might choose according to their taste, some by a quasiobjective system of importance or cultural impact. The room for variation seems almost as large as with five star ratings.

I agree that five star ratings can be very flawed; there’s too much room for human error and not enough nuance. And I’m curious to hear and see more of this pairwise ranking in action. But if you gave me a pairwise ranking of these two options right now, I’m not sure what I would pick.

Your turn for some pairwise rankings: do you like five star ratings or comparative rankings? Tell us in the comments below or on Twitter and Facebook!

IFC_FOD_TV_long_haired_businessmen_table

Hacked In

Funny or Die Is Taking Over

FOD TV comes to IFC every Saturday night.

Posted by on

via GIPHY

We’ve been fans of Funny or Die since we first met The Landlord. That enduring love makes it more than logical, then, that IFC is totally cool with FOD hijacking the airwaves every Saturday night. Yes, that’s happening.

The appropriately titled FOD TV looks like something pulled from public access television in the nineties. Like lo-fi broken-antenna reception and warped VHS tapes. Equal parts WTF and UHF.

Get ready for characters including The Shirtless Painter, Long-Haired Businessmen, and Pigeon Man. They’re aptly named, but for a better sense of what’s in store, here’s a taste of ASMR with Kelly Whispers:

Watch FOD TV every Saturday night during IFC’s regularly scheduled movies.

SAE_102_tout_2

Wicked Good

See More Evil

Stan Against Evil Season 1 is on Hulu.

Posted by on
GIFs via Giphy

Okay, so you missed the entire first season of Stan Against Evil. There’s no shame in that, per se. But here’s the thing: Season 2 is just around the corner and you don’t want to lag behind. After all, Season 1 had some critical character development, not to mention countless plot twists, and a breathless finale cliffhanger that’s been begging for resolution since last fall. It also had this:

via GIPHY

The good news is that you can catch up right now on Hulu. Phew. But if you aren’t streaming yet, here’s a basic primer…

Willards Mill Is Evil

Stan spent his whole career as sheriff oblivious to the fact that his town has a nasty curse. Mostly because his recently-deceased wife was secretly killing demons and keeping Stan alive.

Demons Really Want To Kill Stan

The curse on Willards Mill stipulates that damned souls must hunt and kill each and every town sheriff, or “constable.” Oh, and these demons are shockingly creative.

via GIPHY

They Also Want To Kill Evie

Why? Because Evie’s a sheriff too, and the curse on Willard’s Mill doesn’t have a “one at a time” clause. Bummer, Evie.

Stan and Evie Must Work Together

Beating the curse will take two, baby, but that’s easier said than done because Stan doesn’t always seem to give a damn. Damn!

via GIPHY

Beware of Goats

It goes without saying for anyone who’s seen the show: If you know that ancient evil wants to kill you, be wary of anything that has cloven feet.

via GIPHY

Season 2 Is Lurking

Scary new things are slouching towards Willards Mill. An impending darkness descending on Stan, Evie and their cohort – eviler evil, more demony demons, and whatnot. And if Stan wants to survive, he’ll have to get even Stanlier.

Stan Against Evil Season 1 is now streaming right now on Hulu.

IFC_ComedyCrib_ThePlaceWeLive_SeriesImage_web

SO EXCITED!!!

Reminders that the ’90s were a thing

"The Place We Live" is available for a Jessie Spano-level binge on Comedy Crib.

Posted by on
GIFs via Giphy

Unless you stopped paying attention to the world at large in 1989, you are of course aware that the ’90s are having their pop cultural second coming. Nobody is more acutely aware of this than Dara Katz and Betsy Kenney, two comedians who met doing improv comedy and have just made their Comedy Crib debut with the hilarious ’90s TV throwback series, The Place We Live.

IFC: How would you describe “The Place We Live” to a fancy network executive you just met in an elevator?

Dara: It’s everything you loved–or loved to hate—from Melrose Place and 90210 but condensed to five minutes, funny (on purpose) and totally absurd.

IFC: How would you describe “The Place We Live” to a drunk friend of a friend you met in a bar?

Betsy: “Hey Todd, why don’t you have a sip of water. Also, I think you’ll love The Place We Live because everyone has issues…just like you, Todd.”

via GIPHY

IFC: When you were living through the ’90s, did you think it was television’s golden age or the pop culture apocalypse?


Betsy: I wasn’t sure I knew what it was, I just knew I loved it!


Dara: Same. Was just happy that my parents let me watch. But looking back, the ’90s honored The Teen. And for that, it’s the golden age of pop culture. 

IFC: Which ’90s shows did you mine for the series, and why?

Betsy: Melrose and 90210 for the most part. If you watch an episode of either of those shows you’ll see they’re a comedic gold mine. In one single episode, they cover serious crimes, drug problems, sex and working in a law firm and/or gallery, all while being young, hot and skinny.


Dara: And almost any series we were watching in the ’90s, Full House, Saved By the Bell, My So Called Life has very similar themes, archetypes and really stupid-intense drama. We took from a lot of places. 

via GIPHY

IFC: How would you describe each of the show’s characters in terms of their ’90s TV stereotype?

Dara: Autumn (Sunita Mani) is the femme fatale. Robin (Dara Katz) is the book worm (because she wears glasses). Candace (Betsy Kenney) is Corey’s twin and gives great advice and has really great hair. Corey (Casey Jost) is the boy next door/popular guy. Candace and Corey’s parents decided to live in a car so the gang can live in their house. 
Lee (Jonathan Braylock) is the jock.

IFC: Why do you think the world is ready for this series?

Dara: Because everyone’s feeling major ’90s nostalgia right now, and this is that, on steroids while also being a totally new, silly thing.

Delight in the whole season of The Place We Live right now on IFC’s Comedy Crib. It’ll take you back in all the right ways.