Nnews
Science | DOI: 10.1145/1536616.1536622
Don Monroe
Just for You
Recommender systems that provide consumers with customized
options have redefined e-commerce, and are spreading to other fields.

Sometime soon, A team of pro- grammers is likely to re- ceive a check from Netflix for $1 million. More than 4,000 teams have entered the movie-rental company’s Netflix Prize competition, which was established in 2006 to improve the recommender system Netflix uses to suggest movies to its 10 million-plus customers. As this article went to press, a coalition of previously competing teams, calling itself BellKor’s Pragmatic Chaos, had edged past the 10% ratings improvement over Netflix’s system, which will win them the Netflix Prize (unless another team beats their 10.5% improvement by late July).

VIsualIzatIon by ChrIstoPher hefele

The term “recommender system” has largely supplanted the older phrase “collaborative filtering.” These systems create recommendations tailored to individual users rather than universal recommendations for, well, everyone. In addition to movie recommendations like those from Netflix, many consum-er-oriented Web sites, such as Amazon and eBay, use recommender systems to boost their sales. Recommender systems also underlie many less overtly commercial sites, such as those providing music or news. But in each case a recommender system tries to discern a user’s likely preferences from a frustratingly small data set about that user.

One lure of the Netflix Prize for researchers is Netflix’s database of more than 100 million movie ratings—which include user, movie, date of rating, rating—from some 480,000 users about nearly 18,000 movies. After training their algorithms with this data, teams predict the ratings for a secret batch of 2. 8 million triplets (user, movie, date of rating). Netflix then compares their accuracy to that of its original Cinematch

algorithm. Sharing a massive, real-life data set has energized research on recommender systems, says Bob Bell, a principal member of the technical staff at AT&T Research, and a member of BellKor’s Pragmatic Chaos. “It’s led to really big breakthroughs in the field,” says Bell.

From the start, the Netflix Prize has appealed to academically oriented researchers. The eventual winners, as well as the annual progress prize winners (who receive $50,000), agree to publicly share their algorithms, and many teams openly discuss their research in the online Netflix Prize Forum. For researchers, this openness adds to the

clusters of movies discovered by a computer algorithm created for the netflix Prize competition, with lines closer to yellow representing stronger similarities and colors closer to red representing weaker similarities.