Microsoft develops keyboard capable of swipe & gesture operation with motion sensor

Operation of PC is done with keyboard, mouse and track pad, but it has infrared proximity sensor, it can swipe like smartphone operation, recognize finger gesture and motion and operateMechanical keyboardMicrosoft Research is developing the prototype, which is completed.

Type-Hover-Swipe in 96 Bytes: A Motion Sensing Mechanical Keyboard - Microsoft Research
http://research.microsoft.com/en-us/projects/typehoverswipe/

You can see the movies containing the prototype using the following.

Type-Hover-Swipe in 96 Bytes - YouTube


A prototype of a mechanical keyboard "Type-Hover-Swipe in 96 Bytes" capable of swipe and gesture operation by a keyboard connected to a desktop PC. First let's typing characters.


Then slide the left palm downwards ......


Type position moved to subtitle below.


Furthermore, turning the right forefinger around the keyboard turns the image on the screen.


When you float the left hand on the keyboard and move left and right, the window switches ......


You can scroll the screen by sliding your left hand down.


When you slide your left hand to the left and right, the map moves right and left ......


When you act like pinching with the thumb and forefinger of the right hand on the keyboard, the map is enlarged.


It seems that it can also be used for operation of racing game by turning both hands which are handled as shown below.


The keyboard looks something like this. It looks like an ordinary one.


The way the hand gesture is recognized is displayed on the screen. There is little delay, I also recognize gestures that move hands like a wave or float their hands.


Proximity sensors are installed one above the other in the clearance of the key.


What gesture can be done is "Swipe left and right" that demonstrated at the beginning of the movie ... ...



"Swipe up and down" is available, you can operate like a smartphone.



"Maximize screen" moves your finger to the mountain character on the keyboard.


Pin with fingers and openPinch operationYou can "zoom in and zoom out" on the screen with.



"Search" will start when putting the thumb and index finger at the tip of the keyboard (pinching the keyboard with your fingers) and placing it on the keyboard.


Stand up your fingers and put it in the sky above the keyboard to count the number of fingers.



When the index finger of the right hand placed on the keyboard is lifted up and down, "Index up and down"


Floating the right hand on the keyboard recognizes the top and bottom of the palm.


Turning the index finger around will recognize it as a "loop".


When moving the palm in a wave like manner on the keyboard, it is recognized as a wave gesture.


So I started demonstration using these gestures. First move the finger to the mountain character and maximize the open window.



Slide down to scroll the screen.



After scrolling to the place you want to read, open the fingering finger and zoom in on the screen and browse ... .... That's why it's like a keyboard that can operate like a smartphone.


In addition, it seems that gesture operation using both hands is also possible as follows. Although sales plans etc. are not announced at the stage of prototype now, since operability can be improved without a mouse or a track pad, expectation will be put on future commercialization.