It turns out that the team that won the machine learning contest hosted by Google was hacking the challenge of the competition
by
Kaggle , a subsidiary of Google, provides a platform for engineers and data scientists around the world to compete on how to build the best models for submitted data. A contest at Kaggle revealed that the winning team was building a fraudulent model and winning.
PetFinder.my Contest: 1st Place Winner Disqualified | Kaggle
https://www.kaggle.com/c/petfinder-adoption-prediction/discussion/125436
How a Kaggle Grandmaster cheated in $ 25,000 AI contest with hidden code – and was fired from dream SV job • The Register
https://www.theregister.co.uk/2020/01/21/ai_kaggle_contest_cheat/
In this contest, PetFinder.my, who lists information on animals without access, provided data. The challenge was to build a model that predicts how long it will take to find an animal owner by performing machine learning based on a database containing animal information such as breed, gender, and age. The contest attracted 2,000 teams and the winning team had $ 10,000 in prize money.
by
After examining the machine learning systems submitted by each team, the team led by Kaggle's top ranker Pavel Pleskov won the team by a large margin over the second-place team. I received $ 10,000 in prize money.
It was an engineer who participated in the same contest that noticed the anomaly. When trying to use Pleskov's system that won the first place with other models, the performance dropped considerably. A closer look at the source code shows that Pleskov's system uses loopholes.
The data used to validate the system in this contest was based on the facts published by PetFinder.my, and we were able to know how long it took before the owner was actually found. Pleskov looked at it and built a database by hashing the image data previously published on PetFinder.my and linking it to the time it took to find the owner. If the hash of the image data input at the time of verification matches the hash prepared in advance, a system that returns the linked period is created. Instead of making predictions based on data, Pleskov's system was searching for “facts”, so he achieved very good results.
by kenishirotie
Following this report, Kaggle has frozen Pleskov's account forever. H2O.ai, who works for Pleskov, has also fired Pleskov.
Pleskov apologizes for the fraud on Twitter and states that the $ 10,000 prize will be refunded.
1.We , as a team, would like to apologize to @myPetFinder @kaggle and the DS community for all the wrongdoings.I would also like to apologize to @ h2oai and all the Kaggle competitors for putting their reputation at risk.
— Pavel Pleskov (@ppleskov) January 11, 2020
Related Posts:
