A Microsoft engineer develops a system that manipulates a small drone with just a hand gesture

Sean Kelly, Microsoft software engineer, developed a system to operate a small drone with just a hand gesture. It is possible to intuitively operate the drone just by moving the hand like a conductor. The state of operation by the author Kelly is released in the movie, and the program of this system is released, and anyone can download it for free.

The movie below shows how to manipulate a compact drones with just a hand gesture.

Controlling a Quadcopter with Hand Gestures - YouTube


Sean Kelly, who has released a system for this person to manipulate drone with hand gesture. In the movie that manipulates the drone, Microsoft's head mounted displayMicrosoft HoloLens, A small drone used also in the competition race tournament held on the right sideCrazyflie 2.0 Nano Quadcopterusing.


In order to maneuver the drone, set HoloLens in advance and install it.


Place the powered drone on the floor.


Setting options for HoloLens ......


HoloLens Bluetooth radio on to manipulate the drone!


Now you can control the drones.


When Kelly taps in the air with your finger ......


In the picture of HoloLens, it shows how the scanning of indoor space starts.


When Kelly taps and drags with your fingers, the drones will fly forward and backward, right and left to fit your fingers.


When Mr. Kelly moved his finger, a drone flew toward the screen.


There are two operation methods in this system. The first operation method is an operation method like a radio control, and hand gesture manipulates left and right pitch axis, yaw axis of vertical axis and roll axis of left and right rotation. Kelly is using the flight controller software installed in this droneBetaflightIt explains on the setting screen of.


To operate a drone with a hand gesture, after setting HoloLens, tap and drag by hand. Tap and drag up and down to operate the yaw axis. The blue bar shows the degree of operation of the yaw axis of the drone.


Kelly taps and pulls his finger up so as to draw the thread ... ...


The blue bar showing the degree of operation on the yaw axis gradually grows. Similarly tap and drag forward and backward to manipulate the pitch axis, tap and drag left and right to operate the roll axis.


The second operation is a method in which a drone stationary in the air moves to the fingertip. Tap Kelly and drag ... ...


The drone advances according to the direction you dragged.


According to Kelly, "The most difficult thing to do when making it was where HoloLens let you recognize the position of the hand in three dimensions."


The platform that senses the hand gesture in HoloLens and sends it to the drone isWindows UWPIt is used.


According to Kelly, this drone "Crazyflie 2.0" has a charm that can not be remodeled, and for drones used in movies, optional parts that stabilize the aircraftFlow DeckAs much as incorporating. The program which manipulates this drone is personally made in the vacancy of the private space, and he said that Microsoft technology, program code, etc. are not used. Also, the programSource codeIt is open to anyone, and download is possible.