May 11, 2022 20:00:00

A technology has appeared that allows viewers of game distribution to `` freely stroll through the game being distributed '', and it is also possible to pin up from `` God's point of view ''

Many 3D games allow the player to manually move the viewpoint and explore the in-game world extensively. However, as a matter of course, for those who are just playing the game and watching the commentary video, the world they can see is limited to 'the point of view of the commentator'. We have developed a function that allows viewers to freely view the game world, which may eliminate this common sense.

Enhanced Videogame Livestreaming by Reconstructing an Interactive 3D Game View for Spectators | CHI Conference on Human Factors in Computing Systems

https://dl.acm.org/doi/10.1145/3491102.3517521

Enhanced Videogame Livestreaming CHI 2022-YouTube


Below is what the development team reproduced in the game 'Titanfall 2'. There are three types of screens that viewers can see with the technology developed this time. The first is the normal screen, the screen you see all the time when playing games or watching videos.

The second is like this. It's a projection of the game screen into 3D space, which the team calls 'Volumetric.' Volumetric literally translates to 'volume measurement,' and you can see the word in '

volumetric video technology, ' which converts real space and human movements into 3D digital data.

The third is technology. A 'World' where the viewer can completely leave the player's point of view and freely look around the in-game world. It's not something that's as easy as ``you can look around 360 degrees on the spot'', but it allows you to fly around freely in a 3D space where objects and other things are arranged and look around ``the world where the character is'' from all viewpoints.

Developed by Jeremy Hartmann of the University of Waterloo and others, this technology acquires the data passed from the game to the GPU and converts it into video data, and the video data received through the distribution is converted into 3D space. It consists of two viewer-side applications that build Furthermore, by using data different from the video data, it becomes possible to interact with the distributor and the viewer, for example, ``the viewer puts a marker on the screen the distributor is looking at'' It seems to be possible. .

By wearing VR goggles, it is said that you can get a more immersive feeling as if you were standing in the game, but in any case, you can only reproduce the ``world seen by the distributor''. There is nothing, that is, the place where the broadcaster does not show on the screen becomes a pure white world with nothing. Volunteers have already experienced it, but they commented that there were problems with the quality of objects and shadows flickering on the screen.

Hartman and colleagues will take a larger-scale approach and continue to work to improve quality.