OFF INTO THE WILD WET YONDER How does this...
China Wins! … And Is Promptly Banned for Cheating
China’s Baidu boasted at some length about outperforming both Google and Microsoft in the annual ImageNet image recognition competition. And now it has come out that they were able to do so only by cheating. The company now finds itself banned from ImageNet challenges for a year.
This all began in early May when Baidu posted an article on its tech blog titled “Baidu Achieves Top Results on Image Recognition Challenge.” (The post has since been taken down). Using their new dedicated supercomputer, Baidu stated that it had scored an error rate of 4.58% on the image recognition test, beating out Google’s 4.80% and Microsoft’s 4.94%. For reference, all surpass the 5.10% error rate of an average person). In fact, Baidu got a head start on itself back in January, stating that its supercomputer ‘Minwa’ was the world’s most accurate in terms of image recognition.
What Baidu failed to reveal—and what was subsequently discovered in June—was that the company broke the rules regarding the limit of submissions allowed. The test is similar in concept to Google Photo’s auto-tagging features, and challengers are to submit their best results (in the form of algorithms) that accurately classify objects and context from an enormous database of images.
Only two submissions per week are permitted. Baidu clearly did not get this memo, as challenge organizers have said, “During the period of 28 November 2014 to 13 May 2015, there were at least 30 accounts used by a team from Baidu to the test server at least 200 times.”
There was even a five-day period during March in which 40 submissions were made. Is it 40 > 2? Or is it 2 > 40? Math can be confusing after all. Especially to a room full of computer programmers and IT specialists.
Scrambling, Baidu attempted to write this off as a “mistake,” promising to review the results. Not exactly the most original (or most clever) excuse.
As with most everything in this world, it comes down to money. The applications of image as well as voice recognition and related aspects of “deep learning” have entered a highly competitive (and lucrative) era of development. Numerous companies, small and large, are racing to demonstrate their prowess in this area. Google and Microsoft are already beating humans at image recognition, and this is barely the start.
Overall, this technology goes well beyond applied recognition capabilities and appears to be on the cusp of bringing about even greater advances in artificial intelligence. As such, even fiercer competition can be expected. And with it, fingers crossed, additional scrutiny. Yes, Baidu was caught out this time out. But if we are being fair, all of the big players have had their share of scandal and/or dents put in their reputations. Google included.
Having said that, with China’s less than stellar reputation for copying technology and pushing foreign competition out, it may take Baidu a bit longer than it hopes for to once again be viewed as a legitimate competitor.