Overstock.com runs a $1million contest to find a better recommendation algorithm.
The $1 million Netflix Prize, awarded in 2009, helped the video rental e-retailer find a better method for predicting which movies a Netflix subscriber would like. Now online discount retailer Overstock.com Inc. is offering researchers $1 million if they can develop a better algorithm for recommending products to shoppers at Overstock.com.
Any researcher, or research team, whose algorithm can produce 10% more sales than the recommendations technology Overstock.com currently employs will win $1 million. And if the team works at a university, the e-retailer will donate an additional $250,000 to the educational institution. If no team produces a 10% lift, then the algorithm that produces the biggest increase will win a prorated prize. For instance, an 8% lift will earn a prize of $800,000, Overstock.com said today in announcing the contest.
“This is a phenomenal opportunity to benefit our customers, who will get early, exclusive access to the most advanced recommendations possible through the participation of top educational and research institutions worldwide,” says Overstock.com CEO Patrick Byrne. As part of the contest, Overstock.com will be able to exclusively license the winning algorithm for 18 months.
“The Netflix Prize did a great job of mobilizing the research community around a new and interesting problem,” says Darren Vengroff, chief scientist at RichRelevance and creator of RecLab, the vendor’s testing and research platform. “The RecLab Prize on Overstock.com takes the next step by offering researchers the chance to solve a multi-dimensional, real-world problem and see how their best algorithms perform when put in front of live shoppers.”
Vengroff says he got the idea for the contest last fall when attending an academic conference about online recommendations systems. “A big topic among researchers was how can we get at good data so we can access and solve real-life problems,” Vengroff says.
To test their algorithms, researchers will be given data that’s representative of 75,000 shopper sessions and 139,000 shopper actions, such as looking at an item, performing a search or making a purchase. The data will model real shopper behavior, but Overstock.com will not release any actual shopper data to the researchers, Vengroff says. He says that’s to avoid the kind of embarrassment AOL suffered several years ago when it provided researchers what it believed, incorrectly it turned out, was anonymous data. “Two weeks after they released the data set, the New York Times knocked on the door of a woman in South Carolina and showed her a list of what she was searching for online,” he says.
Teams will have until late fall to submit their entries. Although non-academic teams can enter, the contest is geared toward university researchers and the academic calendar, Vengroff says. “Teams can form, for example, around a class project in the fall semester and then submit it at the end of the semester.”
Semi-finalists will be selected by a panel of judges from Overstock.com, RichRelevance and researchers in machine learning—the study of algorithms that enable computers to come up with increasingly effective solutions based on actual results. The semi-finalist entries will be run early next year against actual Overstock.com data, representing 2% of shopper visits during a week. However, those tests will be conducted within the RecLab facility so that no actual shopper data is released. Three finalists’ algorithms will be run against 10% of Overstock.com data for a week, to determine the winner.
Vengroff notes that contestants will have to improve results compared with the 60 algorithms that RichRelevance makes available to retailer clients like Overstock.com, which has worked with RichRelevance since 2009. Each algorithm is geared to a different shopping scenario. For instance, when a consumer is shopping for DVDs it makes sense to try to persuade him to buy two or three other DVDs. But it makes little sense to try to persuade someone researching washing machines to buy two other washing machines, Vengroff says. In that case, the appropriate algorithm should provide help in finding the right appliance or offer accessories, such as a dryer or hoses needed to install the washer, he says.
The RecLab Prize contest, Vengroff says, “brings in a lot more smart people who work in this area and allows them to apply that brainpower to real Internet retailing problems, much more directly than they’ve been able to do in the past.”
Contest organizers aim to replicate the excitement created by the Netflix Prize, which generated more than 50,000 entries between the time it was announced in 2006 and its conclusion in 2009. A team of AT&T Labs engineers who called themselves Bellkor’s Pragmatic Chaos, won the $1 million prize after demonstrating that their algorithm was 10% better than Netflix’s own method of predicting the films consumers would like.
Overstock.com’s Byrne says he believes a contestant may be able to achieve 10% better results than Overstock.com’s current recommendations technology, and hopes they do. “I would be thrilled,” Byrne says, “if someone got 10% better lift.”