World Labs announces 'AI that generates 3D scenes that can be manipulated from images like a game', and a demo page where you can actually explore 3D scenes is also available



World Labs, an AI development company founded by

Fei Fei Li , also known as the 'Godmother of AI,' has developed an 'AI that generates 3D scenes that can be explored by manipulating the viewpoint like a game from a single image.' A demo page where you can explore the 3D scenes generated by AI was also released, so I tried to see what the AI-generated 3D scene was actually like.



Generating Worlds
https://www.worldlabs.ai/blog

World Labs' AI can generate interactive 3D scenes from a single photo | TechCrunch
https://techcrunch.com/2024/12/02/world-labs-ai-can-generate-interactive-3d-scenes-from-a-single-photo/

Regarding the newly developed AI, World Labs said, 'We are working on building an AI system that generates 3D worlds from a single image.' 'Given an input image, our system estimates 3D geometry, fills in unseen parts of the scene, creates new content so it can be oriented, and generalizes to a variety of scene types and art styles.'

There are already some AIs that can generate 3D models and 360-degree images from images, but World Labs' AI is unique in that it can move the viewpoint of the 3D scene like a game. The output 3D scene can be rendered in real time in the browser with full camera control.

In fact, World Labs has released a demo page where you can operate the 3D scene generated by their AI. If you access the demo page and scroll down, you will see the following demo screen.



Click and drag on the demo screen and the camera will move in the direction you drag.



The camera can also be tilted up and down.



The viewpoint can be controlled by keyboard, moving forward with 'W', to the left with 'A', to the right with 'D', and to the back with 'S'. However, the distance you can move is not very far, and after a little while, 'Out of bounds' is displayed and you can no longer move.



There are some areas where the rendering is poor, but overall the quality is quite good and there is a sense of unity.



Click on an image at the bottom of the demo screen to move that image's 3D scene.



There was also an aerial view.



Scrolling further down the screen, a demo screen appeared in which

the depth of field could be adjusted to focus only on a specific object.



By adjusting the depth of field with the slider above and moving the camera with 'W', 'A', 'D', and 'S', the viewpoint changes while keeping the subject clearly in focus.




Further down, there is a demo screen that can reproduce a technique called

dolly zoom, in which the background zooms in or out while the size of a specific subject remains the same.



By moving the slider below, you can see how the background changes without changing the size of the subject.



The demo page also allows you to recreate animation effects in the 3D scene, such as 'clicking on a specific object to shine a spotlight on it' and 'creating ripples on the surface of an object,' and you can also explore the world of famous paintings in 3D scenes.



TechCrunch, a technology media outlet, pointed out that previous AIs that generated 3D scenes from images had the drawback of low scene resolution and quick changes in layout. 'In contrast, World Labs' approach allows the scene to remain constant even after generation and follows basic physical laws, giving it a sense of three-dimensionality and depth,' the company said.

'3D world generation can be combined naturally with other AI tools, allowing creators to create new experiences using the tools they're already familiar with,' World Labs said, revealing that it has already rolled out the AI to a few creators.

You can sign up for the World Labs waitlist via the following Google Doc:

World Labs waitlist
https://docs.google.com/forms/d/e/1FAIpQLSf9jHsaDq1IwM_FADQP0Gbd82tbW4CBOI5YfUAdPfqrFrWEeA/viewform

Related Posts:

in Review,   Software,   Web Service, Posted by log1h_ik