Professor MIT developed automatic document creation software to prove incompetence of 'software to automatically judge superiority / inferiority of sentences'

ByJD Hancock

Software which automatically judges superiority and inferiority of the sentence before less than 1 second by inputting English sentences has been developed, and it is utilized in the evaluation of the problem report at the site of some higher education in the United States It is. However, scholars who are skeptical of such sentence judgment software themselves make software that automatically generates "beautiful English sentences" themselves, and let sentence judgment software judge the sentences automatically generated, so that the sentence judgment software It demonstrates whether it is incompetent and usable.

Writing Instructor, Skeptical of Automated Grading, Pits Machine vs. Machine - Technology - The Chronicle of Higher Education
http://chronicle.com/article/Writing-Instructor-Skeptical/146211/

Professor Les Perelman of MIT had a skeptical view on the ability of software called EASE to judge document superiority and inferiority for a long time. And he seems to have felt that the EASE was going to wake up the current situation being utilized at the educational site to evaluate the merit of the issue report.

Therefore, Dr. Perelman decided to create a document generation software "Babel Generator" that can automatically generate "beautiful English sentences". Babel Generator is a software that automatically generates long sentences containing the word simply by entering an appropriate English word, and the syntax of the sentence is perfect. However, the sentences that the Babel Generator automatically generates in less than one second are designed to simply be a tasteless drying with just the perfect syntax and completely no content.

Dr. Perelman and researchers.


Dr. Perelman entered one keyword "privacy (privacy)" to automatically prepare a sentence and made the sentence judged as an online test of adopting EASE called MY Access! 5.4 points full score of 6 I hit a high score. Of course, this privacy sentence is a stuff with no content.

Experiments to examine the accuracy of sentence judgment software are done besides Dr. Perelman. Professor Daniel Bornebach of the University of Texas,MOOCI read 100 essays of students taking philosophy at philosophy and programed to imitate the evaluation style of Professor Bornback. And when the student's report submitted at the end of the term was evaluated by Professor Bornback on sentence judgment software, it seems that he gave a very close evaluation to Professor Bornback. However, it was said that A (excellent) was attached to the report that should be judged as C (acceptable).

Also, in cases where we tested sentence judgment software besides Professor Bornback, we found an example that gives low evaluation to excellent reports. Although the text judgment software generally gave a reasonable evaluation, it turned out that it tends to put a low evaluation on the sentences of students who are not native English speakers. Professor Bornback evaluates "Writing judgment software is still far from practical use".

ByTom Keene

In addition, Dr. Perelman does not deny the significance of the technique to judge the superiority or inferiority of the sentence more quickly and accurately than a human being, but it does not deny the significance of a technique to quickly and accurately judge the superiority or inferiority of a sentence, and he expects such superior technology to be realized someday. However, at the present time, I conclude that the accuracy of sentence judgment software is not high.