The call for an audit of Facebook’s metrics comes a week after the social network acknowledged inflating its video metrics.
Years after challenging anyone to top by 10% the accuracy of its system that guesses ratings for movies members have not seen, Netflix has announced BellKor’s Pragmatic Chaos has laid claim to the prize. Other teams now have 30 days to top Chaos.
Two-and-a-half years after challenging anyone to beat by 10% the accuracy of its system that guesses how members will rate movies they’ve not yet seen, Netflix Inc. has announced on its Netflix Prize Leaderboard that a team known as BellKor’s Pragmatic Chaos has laid claim to the $1 million prize.
Per the rules of the contest, this starts a ticking clock that gives other teams 30 days to try and beat BellKor’s Pragmatic Chaos’ improvement factor of 10.05%.
The Netflix Prize seeks to substantially improve the accuracy of predictions on how much members are going to like or dislike a movie they have yet to see based on their ratings of movies they’ve already seen. Netflix members rate movies on a scale of one to five stars. The more movies they rate, the easier it becomes for Netflix, No. 18 in the Internet Retailer Top 500 Guide, to guess how members will rate movies they have yet to see.
The BellKor’s Pragmatic Chaos team has submitted evidence that the predicted ratings its algorithms generate are 10.05% more accurate than those generated by Netflix’s current ratings prediction system, called Cinematch. If no other team matches or exceeds 10.05% before the 30-day period is up, then Netflix will initiate a validation process it says will take a few weeks. If its system is validated, BellKor’s Pragmatic Chaos will be awarded the $1 million prize.
BellKor’s Pragmatic Chaos is actually a team of teams. Four teams that have been working separately during the past two-and-a-half years joined forces under the BellKor’s Pragmatic Chaos moniker. Team members include:
- Bob Bell and Chris Volinsky, members of the statistics research department at AT&T; Research, and winners of 2007 and 2008 Netflix $50,000 Progress Prizes as members of the team BellKor.
- Andreas Toscher and Michael Jahrer, machine learning researchers and founders of Commendo Research and Consulting in Austria. They won a 2008 Progress Prize as members of the team Big Chaos.
- Electrical engineer Martin Piotte and software engineer Martin Chabbert of Montreal, founders of Pragmatic Theory.
- Yehuda Koren, senior research scientist at Yahoo Research Israel, and winner of 2007 and 2008 Progress Prizes as a member of team BellKor.