Researchers announce technology 'FingerTrak' to track hand movements only with wristband type device
Technologies related to
FingerTrak: Continuous 3D Hand Pose Tracking by Deep Learning Hand Silhouettes Captured by Miniature Thermal Cameras on Wrist: Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies: Vol 4, No 2
https://dl.acm.org/doi/abs/10.1145/3397306
Researchers develop 3D hand-sensing wristband | Cornell Chronicle
https://news.cornell.edu/stories/2020/07/researchers-develop-3d-hand-sensing-wristband
Researchers show FingerTrak, a hand tracking wristband for AR/VR input | VentureBeat
https://venturebeat.com/2020/07/20/researchers-show-fingertrak-a-hand-tracking-wristband-for-ar-vr-input/
When playing VR games, it is often the case that a stick-type controller with a tracking function is held, but it is difficult for a general controller to track finger movements related to various daily activities. For example, movements such as writing characters with a ballpoint pen, holding a mug, or operating a smartphone cannot be tracked while holding the controller.
Therefore, in recent years, development of devices that track even finger movements has progressed one after another, and Sony announced ' Next-generation VR game controller that can track finger movements with high accuracy ' and ' Sign language via smartphone Researchers have developed a ' glove-type device that can be converted to voice with .'
Cornell University's research team has newly developed a wristband type device ' FingerTrak ' that tracks the movement of the finger without using gloves or a ring fitted to the finger. You can see the shape and mechanism of FingerTrak developed by the research team by watching the following movie.
FingerTrak Continuous 3D Hand Pose Tracking by Deep Learning Hand Silhouettes-YouTube
Accurate tracking of finger movements is an important and popular topic for VR and AR advances.
Glove-type devices that track finger movements have already been developed, but their bulkiness and poor portability can hinder VR and AR immersiveness.
Therefore, the research team developed 'FingerTrak', a wristband type device that tracks finger movements. Compared to a 25-cent coin with a diameter of 24.26 mm, the size looks like this. You can see that there are 4 cameras on the wrist side.
The tip has a low-resolution infrared camera that detects the temperature of the object. The length of the camera is 5.7mm……
The diameter is only 9.3 mm.
FingerTrak tracks the contours of hands and fingers in 3D with an infrared camera directed toward the hand, and uses deep neural networks to infer the positions of fingers and wrist joints from the contours.
The hand contours obtained from the four cameras are very messy, but research team Cheng Zhang said, “This was a big discovery for our team. We can reconstruct the movements of the fingertips with high precision in 3D.'
When the wearer moves his finger...
In the infrared camera image displayed at the bottom left, the range shown in orange or yellow changes. Based on this change, it is possible to measure the movement of the fingertip with considerably high accuracy. According to the research team, the error rate of the finger angle detected by FingerTrak was 6.46 to 8.06 degrees, and the more uniform the background, the lower the error rate is, the research team says.
By using FingerTrak, you can hold a pen and write something in a notebook.
Movement with mug
It is possible to identify movements such as holding a smartphone in your hand and operating it. The research team believes that FingerTrak is useful for sign language translation technology, and by tracking the movement of the hand of the elderly, it may be applied to detection of diseases such as Alzheimer's disease and Parkinson's disease that show impaired fingertip movements. I don't know.
The prototype FingerTrak has a wired connection to the Raspberry Pi, but it can also be connected to and controlled by a computer such as a smaller smartwatch.
Related Posts:
