Artificial intelligence (AI) marks a score that surpasses human beings at the Stanford University reading test

byMarkus Spiske

In January 2018, Alibaba gave 100,000 questions to artificial intelligence (AI) to test the performance of its proprietary deep neural network model and tested whether it can derive an accurate answer. This "question given to AI" is a data set for AI reading test created by Stanford University "SQuAD"In this test Alibaba developed AI blew out a better score than humans.

Alibaba's AI Outguns Humans in Reading Test - Bloomberg
https://www.bloomberg.com/news/articles/2018-01-15/alibaba-s-ai-outgunned-humans-in-key-stanford-reading-test

Alibaba neural network defeats human in global reading test | ZDNet
http://www.zdnet.com/article/alibaba-neural-network-defeats-human-in-global-reading-test/

SQuAD's question based on articles in Wikipedia exceeding 500 is made to clarify whether machine learning model handles a large amount of information and can provide accurate answer to question is. By testing the reading performance of AI using this SQuAD, it is possible to evaluate how much reading performance the AI ​​has with the score. Various research institutes and IT companies such as Google, IBM, Microsoft, Samsung, Tel Aviv University, Kangwon National University participated in this SQuAD test. Among them, "SLQA +" of AI developed by Alibaba has recorded a score of "82.44".


According to Alibaba, this time it is the first time that SLQA + exceeds the human score (82.304). Two days before Alibaba announces the score of SLQA +, Microsoft's AI "r-net +" also records "82.650" which is a score higher than human.

The score of other participants can be confirmed from the following page.

The Stanford Question Answering Dataset


In the neural network model used by Alibaba, "read from paragraph to sentences, words" to identify the answer "Hierarchical Attention NetworksIt is based on the technology called "technology."

In fact, this technology seems to have been used to respond to inquiries of "single day" when total transaction amount reached 168.2 billion yuan (about 2.7 trillion yen).

The total transaction amount of online shopping event set by Alibaba in China reached less than 3 trillion yen on "Single Day" - GIGAZINE


Alibaba is entering AI development competition like Tencent and Baidu of the same Chinese. We hope that China will lead the industry in the AI ​​industry by 2030 and we are doing a lot of support for AI related companies for that. Among the fields that make use of such AIs, things like "SLQA +" trying to understand human languages ​​"are classified in the field of natural language processing.

Luo Si, who is conducting research on natural language processing at Alibaba, said that "This technology (SLQA +) is beginning to gradually be adopted in a number of applications, including customer service, museum tutorials, online response to medical inquiries from patients , We can reduce the need for input by humans in an unprecedented way. "