Airbus and JAL jointly developed mixed reality training equipment that spreads real cockpit in front of Hololens

Microsoft's holographic head mounted display realizing mixed reality (MR) "HoloLens"To train pilots, mechanics, crews by projecting the cockpit and aircraft like the real machine in front of youMixed reality training applicationAirbus and JAL jointly developed. On November 14, 2017, the prototype was released for the press.

Airbus and JAL have been developing blended reality training equipment using Hololens since 2016. In the prototype released this time, the A350 XWB aircraft's body, cockpit, door portion, etc. are actually displayed in front of you, so you can train the procedures necessary for operating the aircraft by using hands and controllers It is becoming.


On the day, a demonstration using Hololens was actually demonstrated. The scenery of the cockpit shown on the back screen is displayed inside the Hololens, and the user can learn the operation procedure as though it is operating a real fuselage. By the way, the projector's display has a time lag somewhat than the actual operation, but there is almost no deviation in the video displayed on Hololens.

Where to operate the cockpit with a training system using Hololens jointly developed by Airbus and JAL - YouTube


The scenery visible inside Hololens is such an image. In the eyes of the demo performer, in addition to the real landscape, panels and displays of cockpits that can not exist there are displayed.


Turning around the head and looking around, the cockpit of the A350 XWB aircraft has been fully reproduced. The shape of each part is provided with genuine 3D data from Airbus, and the scenery which is not different from the actual machine is reproduced, and each button can be operated to perform the same operation as the actual machine.


The aircraft radio which communicates with the control is fully reproduced, and it is possible to actually input the frequency. The contents entered are of course reflected on the screen of the instrument.


Adjusting engine power, two thrust levers can also be operated.


On the screen of the simulator, you can see a display for sending compressed air "bleed air" for starting the engine. In a mixed reality training application in which even such detailed procedures are reproduced, it is possible to train anytime, anywhere, and immediately by tracing Hololens's simulation, but exactly the same procedure as the actual machine .


The joint development of JAL and Airbus this time is realized from JAL starting operation of Airbus A 350 XWB type machine from 2019. Airbus is selected as one member of Microsoft's "mixed reality partner program", and we are developing mixed reality application using Hololens. Among them, by combining JAL with the introduction of the first Airbus machine, it is said that it utilizes feedback from the actual aircraft operator for innovation.


In this demonstration, in addition to cockpit training, a program for mechanics was also released. Demonstration was done program to train the procedure to open the door from outside the machine. Even if you just open the door, there are several procedures defined for the aircraft.


First, check the display showing the pressure difference with the inside of the aircraft, then operate the lever that opens the door. An example is shown on the Hololens screen with a light blue color, "Take such a pose." First of all, put one hand on the door ... ...


With the other hand I pulled the handle embedded in the door, the door opened. Of course it is also possible to train the procedure to close the door. Ultimately there is no difference in necessity of training using the same model as the real machine or the real thing, but it is possible to practice and review everywhere at any time if there is only Hololens, so that training more efficiently It will be possible.


Apart from the mixed reality of mixed reality Hololens, it was equipped with a liquid crystal displayLenovo ExplorerHowever, it is possible to use the same application. Rather, the Lenovo Explorer, where the scenery fully spreads before the eyes, is more immersive than the Hololens which is not wide in the display area.


A demo at Lenovo Explorer looks something like this. On the screen of the notebook PC, the scene seen by the headset is displayed as it is, but you can see how the display quickly follows the movement of the head.

Training the procedure to start the A350 XWB aircraft engine with Hollens virtual reality - YouTube


I tried a little mixed reality training application, but I remembered the illusion of sitting in the cockpit of a real A350 XWB aircraft. When the button was operated, the state that the corresponding part reacted was also reproduced as it was, and it seemed that it was possible to perform interactive training. Of course, at the end there is still a need for training on real machines, but there is a great possibility for a system that allows easy and full-scale training at the stage of progressing to that point That's right.

Combined reality training application using Airbus and JAL's Hololens aimed at being able to practice training anytime, anywhere, many times, and cheaply. It is said that the plan to be actually introduced to the training site is yet to be determined and development is going to be advanced in the future, but Airbus, which led the development, is interested in this system, others Several similar projects are also being carried forward in the aviation field.