'Toon3D' recreates 3D space from a few 2D images

A tool called ' Toon3D ' has been released that recreates 'plausible 3D structures' from a few hand-drawn images of manga and anime.

Toon3D

Humans can recognize the 3D world from 2D images, but machines are not good at this. 'Toon3D' is a tool that uses multiple images to complement the space not depicted in the images.

Below is a sample video.



The sample images used are below.

The other one is below. Based on these two images, Toon3D recreated the space.

The recreated scene is blurry. The demo used two images, but it seems that three or more images are needed.

Here's another sample video:



Even hand-drawn paintings can be reconstructed with Toon3D, which predicts the depth of each image, aligns and warps the point clouds, and uses Gaussian refinement to create a video like this:



Toon3D is also available on GitHub.

GitHub - ethanweber/toon3d: Code for Toon3D https://toon3d.studio/
https://github.com/ethanweber/toon3d?tab=readme-ov-file