Apple announces the AR headset 'Vision Pro', released from the restrictions of the display and can operate your favorite applications with your eyes, hands and voice wherever you like

Apple has announced its first AR headset, the Vision Pro . A fully three-dimensional interface is adopted, and operation is possible only with the eyes, hands and voice.

WWDC23-Apple Developer
https://developer.apple.com/jp/wwdc23/

WWDC 2023 — June 5 | Apple-YouTube


Apple's AR device 'Vision Pro' has appeared.

It looks like this when Vision Pro is installed.

Circular icons line up on the home screen.

When you open the app it looks like this.

Various operations are performed with 'eyes', 'hands' and 'voice'.

Apps can be freely placed in space.

It can also be displayed so as to surround the user.

When you wear Vision Pro, the surrounding scenery is first displayed as a pass-through.

Then you will see the home screen.

Various interfaces have a three-dimensional effect.

You can freely adjust the size of the app.

You can change the position freely.

Since the apps are automatically arranged, the situation where 'apps overlap and are difficult to see' does not occur.

Instead of passing through the surrounding scenery, it also has an 'environment' function that displays arbitrary scenery.

The degree of blending between the pass-through display and the 'environment' can be adjusted with the Digital Crown.

Vision Pro can be operated with 'eyes', 'hands' and 'voice'. You can control the focus by moving your line of sight.

Once you focus on the target with your gaze, you can select it by 'tapping between fingers'.

You can scroll by moving your finger up and down.

It is possible to operate in a relaxed posture like this.

If you match your gaze with the input field, voice input is possible.

You can also use Siri to launch apps and control media playback.

When it detects that there are people nearby, the 'user's eyes' are projected outside the Vision Pro. Apple calls this feature 'EyeSight'.

When you are watching content intensively, your eyes will not appear outside the Vision Pro, giving you the information that you can't see your surroundings.

Also, when someone approaches, the figure of the person is projected on the screen of Vision Pro.

The display is very high definition and you can read fine print without any problems.

Vision Pro can be used by connecting a mouse and keyboard via Bluetooth. This makes it easy to use email and spreadsheet apps.

Furthermore, if you look at the Mac with Vision Pro attached, you can display the 'Mac external display' on the Vision Pro screen.

This allows you to use the Mac app and the Vision Pro app at the same time.

Video calls show the person you are talking to in life size.

Also, when you open a photo, the surroundings of the photo are automatically dimmed.

It is also possible to appreciate panoramic photos on an overwhelming scale.

In addition, Vision Pro can be used as a 3D camera. You can record the subject as a photo or movie with 3D information.

Playing 3D videos will make you feel like you are looking into your memories.

Vision Pro is also excellent as a video content viewing environment. First, it is possible to make a huge screen appear in the field of view.

You can also dim the surroundings while watching content.

It also supports spatial audio, so it has an excellent sense of immersion.

Furthermore, you can use the 'environment' function to create an environment that follows the video content.

A 'cinema environment' that reproduces a movie theater is also available.

3D images can also be viewed stereoscopically.

Furthermore, it is also possible to play games by connecting a controller for Xbox or PlayStation.

Over 100 games will be playable when Vision Pro launches.

Disney CEO Bob Iger appeared during the announcement of Vision Pro.

It was announced that Disney's streaming service 'Disney +' will be available when Vision Pro is released.

The front of the Vision Pro is made from a single piece of 3D molded glass.

A large number of cameras and sensors are installed inside the glass.

The frame has buttons and a Digital Crown for playing 3D photos and videos.

The frame is made of aluminum alloy and is curved to fit the shape of your face.

It uses a modular system where each part can be replaced with different sizes to ensure a perfect fit for everyone.

A wide range of shading pads are available.

The headband is also removable and replaceable.

We have also developed corrective lenses in cooperation with ZEISS for spectacle wearers. The corrective lenses are magnetic and easy to attach.

Vision Pro can be used by connecting to a power supply.

It can also run for up to 2 hours with an external battery.

The screen display uses a micro OLED.

The total number of pixels on the two panels is 23 million.

The eyepiece consists of 3 pieces.

The combination of a panel with a total of 23 million pixels on the left and right and a uniquely designed lens realizes a high-definition screen that allows you to see fine characters.

Audio is played through an 'audio pod' with integrated dual drivers.

A technique called 'audio ray tracing' is used to analyze the characteristics and materials of the surrounding space and reproduce sounds that match the user's space.

There are two main cameras on the front.

In addition to the main camera, it also has a downward camera, an infrared illuminator, and a side camera.

In addition, an infrared camera is installed near the eyes to detect the movement of the line of sight.

Vision Pro is equipped with two types of processing chips to achieve 'low temperature and quietness' and 'high processing performance'. The first chip is 'M2' and can operate almost silently.

And another chip is 'R1' specially designed for real-time sensor processing.

The R1 processes input from 12 cameras, 5 sensors and 6 microphones in real time.

R1 sends processing from 'camera' 'sensor' 'microphone' to the display within 12 milliseconds. As a result, it is possible to reduce phenomena such as 'VR sickness'.

It is also possible to create a digital persona using the sensors installed in Vision Pro. Digital Persona will be used for video chat and other purposes.

Vision Pro is equipped with the newly developed OS 'vision OS'.

vision OS is designed from the ground up for spatial computing.

Some developers have already been provided with a development version of Vision Pro, such as 'an application that displays an interactive 3D heart model' and 'an application that confirms the design concept of an F1 machine'.

Vision Pro can also use Microsoft applications such as Excel and Word.

You can also use video conferencing apps such as Teams and Zoom.

For developers, a toolkit 'Reality Composer Pro' will be provided that will allow them to utilize 3D space.

Also, the framework that can be used with iPadOS and iOS can be used as it is with Vision Pro. With this, a huge number of apps for iPad and iPhone are also available for Vision Pro.

For example, you can enlarge the screen of Adobe Lightroom and operate it with your eyes and hands.

In addition, it also supports the game engine 'Unity', allowing you to utilize the 3D functions of Vision Pro in your games.

In the App Store for Vision Pro, you can download apps for iPad and iPhone in addition to dedicated Vision Pro apps.

Vision Pro is unlocked by 'Optic ID' that reads the iris.

Vision Pro uses gaze input, but gaze information is not notified to the app or website.

The Vision Pro is priced at $3,499 (approximately 488,000 yen) and will be available on Apple's official website and Apple Store in the United States in early 2024.

It will also be available in countries other than the United States by the end of 2024.