The New York Times Magazine published an interesting article that discusses the Netflix $1mn challenge that has some developers working tirelessly to improve the accuracy of the Netflix recommendation engine by 10%.
The company’s Cinematch technology is designed to introduce Netflix members to new movies that they’re likely to enjoy. By 2006, the engineers at Netflix were unable to improve Cinematch’s performance.
This posed an interesting challenge for the company because a substantial portion of its rentals were being driven by Cinematch. Today the company attributes a whopping 60% of its rentals to Cinematch recommendations. Most importantly, the company learned that Cinematch is driving more rentals, which helps Netflix retain its subscribers.
So what to do?
Netflix embarked on a crowdsourcing challenge – boost the performance of Cinematch by 10% and we’ll pay you $1mn.
Today, 30,000 aspiring millionaires are trying to hit that elusive 10%. They’ve downloaded a list that contains 480,189 member ratings across 17,770 Netflix movies. No personally-identifiable or demographic data is provided.
The challenge has attracted a wide array of contenders – from Ph.D. computer scientists to self-taught hackers.
If anything highlights the importance of recommendations to online retailers, it’s the Netflix challenge. While $1mn may seem like a hefty amount to pay for “only” a 10% improvement in a recommendation engine that already works quite well, Netflix knows that a 10% improvement will generate far more than the $1mn it has to pay.
In his article, which is well worth the read, the New York Times’ Clive Thompson does a great job of taking a broad perspective of the online recommendation engines that are now so pervasive.
As Thompson points out, before the internet, if you needed a recommendation, you usually asked a friend, a family member, an acquaintance, a clerk. Or, of course, you relied on your judgment.
“The advent of online retailing completely upended this cultural and economic ecosystem,” Thompson states.
Thanks to the fact that websites can track just about everything, it’s possible for online retailers to come up with increasingly sophisticated ways to mine the vast amounts of data they collect to spot patterns that we could never detect on our own and to make recommendations that are quite accurate much of the time.
But as Netflix has learned, there do appear to be limits. The leader in the Netflix challenge has boosted the performance of Cinematch by 9.44%. Close, but no cigar.
According to Len Bertoni, who is a top 10 contender, the enemy of the Netflix hackers might be a single movie – the cult hit Napoleon Dynamite.
As Thompson observes:
“…’Napoleon Dynamite’ is very weird and very polarizing. It contains a lot of arch, ironic humor, including a famously kooky dance performed by the titular teenage character to help his hapless friend win a student-council election. It’s the type of quirky entertainment that tends to be either loved or despised. The movie has been rated more than two million times in the Netflix database, and the ratings are disproportionately one or five stars.”
“Worse, close friends who normally share similar film aesthetics often heatedly disagree about whether ‘Napoleon Dynamite’ is a masterpiece or an annoying bit of hipster self-indulgence.”
According to Bertoni, Napoleon Dynamite accounts for 15% of his remaining “error rate.” Given that he has been able to boost Cinematch performance by 8.8%, the Napoleon Dynamite “factor” is all that separates him from $1mn.
As I read Thompson’s article, I couldn’t help but think two things:
It’s really quite amazing just how far we’ve come with recommendation engines like Cinematch. Although we take it for granted, the idea that we can now fairly accurately predict how well individuals will like, for instance, a movie, based completely on quantitative data alone is pretty amazing. The idea itself is not absurd (it’s quite logical) but the success of practical implementations of it is really quite something.
- There are, however, limits to recommendation engines. It may be possible for Cinematch to predict that someone who liked The Usual Suspects would likely enjoy Reservoir Dogs but quirky films like Napoleon Dynamite have more subtle, almost ethereal qualities that elude quantitative analyses.
We should appreciate the ability of computers and algorithms to add value to our lives (and to boost our businesses). At the same time, there are some things that just can’t be explained by computers and algorithms.
I would suggest that we don’t want computers to become so efficient at predicting our preferences and tastes that we never rent a bad movie again.
While I love technological innovation, there’s a part of me that hopes the Netflix challenge won’t be met. Perhaps if it isn’t, it will have been the universe’s way of telling us that our consumer culture isn’t Newtonian in nature. That we can’t mechanically predict what we’re going to like or dislike. That there’s an element of unpredictability that makes our lives a little bit more interesting and a little bit richer.
After all, if everything we consumed met our expectations, what fun would like be? What life-changing experiences would we miss out on? How would we grow? How would we learn? How would we shape our identities?
I find it somewhat ironic that society often pays the most attention to individuals who are dynamic and hard to “quantify.” We generally have an appreciation for unpredictability. Who wants to do the same thing every day? Who wants to become so static that his friends and family can deduce everything about who he is and what he likes?
We’re told that we should seek out new experiences, that we should try to learn something today that we didn’t know yesterday, that we should challenge our beliefs. Indeed, we’re taught to appreciate the fact that people can change.
Yet if we constantly endeavor to ensure that we never watch a movie we won’t like, read a book we won’t enjoy or listen to a song that hurts our ears, how will we ever maximize our potential as human beings?
While I think recommendation engines serve a very valid business purpose and certainly wouldn’t recommend that online retailers do without them, the search for a perfect recommendation engine is really the search for a “positive feedback loop” that will ensure that we never consume anything we know we won’t like because we’re never exposed to anything that we might learn we like.
Bad movies are a part of life. While I really didn’t really enjoy Snakes on a Plane, I will admit that I did smile once or twice. And who was ever harmed by a smile?