Microsoft is developing technology to translate voice and sign language in real time using Kinect

ByJung-nam Nam

I can choose the perfect jeans without trying on it,You can transform into Ultra Seven,FurthermoreBy adopting it at the site of surgical operation, the work burden of the surgeon can be greatly reducedAnd gesture tracking system active in various applications Kinect. Using this Kinect, Microsoft is developing a translation system "Kinect Sign Language Translator" that enables translators to translate sign language into speech language in real time for those with impaired hearing and vice versa is.

Microsoft Research uses Kinect to translate between spoken and sign languages ​​in real time - The Next Web
http://thenextweb.com/microsoft/2013/10/30/microsoft-research-uses-kinect-translate-spoken-sign-languages-real-time/


You can actually see the translation of sign language content in real time using Kinect in the following movie.

Kinect and sign language translation - YouTube


Kinect captures gestures, interprets the meaning of sign language through machine learning and pattern recognition programming, and actual language translation is done by Bing's translation technology. This Kinect Sign Language Translator can translate sign language into speech or translate speech into sign language.

Do sign language ......



Kinect will capture that movement.



And when the sign language ends, it translates into voice and letters.


Enter characters ......


Click "SEND".


Then the 3D model will convey the contents of the letters in sign language.



Furthermore, using the voice recognition function ......


Translate the voice content and express it in sign language.



Currently, five people are required to establish a recognition pattern for one word, so far it seems that 300 words out of the total 4000 words of Chinese sign language are added as a recognition pattern. Guobin Wu, program manager for the Kinect Sign Language Translator project,Data glovesOr trying to translate sign language using webcam, Kinect was adopted as the terminal best suited for this translation work. Mr. Wu estimates that more than 20 million people with impaired hearing are in China, over 360 million people worldwide, and as this research will save many people all over the world, It seems to be thought.