Facebook introduces 'Rosetta' which AI interprets and extracts aggressive messages of 'Internet Meme'

Among the Internet Meme ( internet meme ) which let letters spread on still images and GIF animation images, there are politic things that invite laughter, and meme that attack others. On Facebook, it was announced that the system " Rosetta " capable of detecting such a malicious meme using AI was developed and introduced.

Rosetta: Understanding text in images and videos with machine learning - Facebook Code
https://code.fb.com/ai-research/rosetta-understanding-text-in-images-and-videos-with-machine-learning/

Facebook is making AI that can identify identity offensive memes - The Verge
https://www.theverge.com/2018/9/11/17846512/facebook-rosetta-meme-machine-learning-text-recognition-hate-speech

Rosetta introduced by Facebook is a large-scale system in which AI analyzes the frames of images and video files that exist on Facebook and umbrella Instagram, and finds malicious messages written there. Traditionally it was possible to extract characters from images using OCR technology , but we were not able to understand the meaning indicated by that character. Rosetta allows you to extract characters from images and videos and find Internet mime that seems to contain a problem by understanding the meaning of that letter. Its ability is that it can handle more than 1 billion data per day.

Rosetta is roughly divided into two parts, "Detection" and "Recognition". Detection uses Faster R-CNN which developed the convolution neural network (CNN), "Proposal of reliability score (Proposal)" in the image which character seems to be included → "Judgment on character" → Extract the characters included in the data by "Extracting Characters" procedure.

Recognition uses a CNN based model of ResNet 18 architecture, and a mechanism to recognize characters as words is used.

In this model, training of recognition using a short word of about five characters was done first, then training of 6 characters, 7 characters and so on is being expanded.

Rosetta also supports languages ​​other than English, it is announced that it supports languages ​​such as Spanish as well as Arabic languages.

By using this mechanism on Facebook, we are aiming to exclude content contrary to our policy that prohibits hate speech from the platform. On the other hand, The Verge points out the possibility of grasping users' concerns by extracting information from images with the same mechanism and reflecting them in news feeds and the like.