Technique to make object reflected in image into 3D and move around

"3D Object Manipulation in a Single Photograph Using Stock 3D Models" (3D Model Data for Existing 3D Model) can be reconstructed from three-dimensional objects of two-dimensional data such as photographs and paintings later and placed at desired positions and angles on the screen 3D object manipulation on the single image used) has been developed.

3D Object Manipulation in a Single Photograph using Stock 3D Models
http://www.cs.cmu.edu/~om3d/

Using this technology, small folded crane placed on fingers ... ...


It has become possible to obtain surprising effect of starting flapping and flapping.


Details of that technology are explained in the following movie, and it shows how the image moves like a magic.

3D Object Manipulation in a Single Photograph using Stock 3D Models: Full Length Video


A picture of a certain room. I see a red chair with a single leg, but apparently this will start to move.


However, if it is a normal method, it cuts a part of the image and moves it, or it finishes processing such as enlargement / reduction and angle rotation.


However, in the real world to move like this ... ...


It can add motion such as turning over.


It is the technology of "3D object manipulation" that enables such movement on image data. By combining the photograph and the 3D model data, it was possible to move the image which was originally a 2 dimensional image to 3D.


The 3D model can be used in a general form, and in the movie IKEA'sIKEA Home PlannerData that can be downloaded from is used.


Here is the image of overlying 3D data. However, I feel that it is not blended with the background.


When texture mapping picture images to 3D data of chair drawn with wire frame, it will be natural finish ......


Of course, the parts that were not visible in the picture can not be reproduced.


3D object manipulation technology is to reproduce natural 3D data by combining original image, 3D data, and light source in combination.


From the left, if you look at the original 3D data, the image mapped state, and the completed form after processing, the difference is obvious.


Dedicated software is used for processing. First of all, in order to define a three dimensional space, draw two reference lines for each of length and width.


In this example, the wire frame is adjusted based on the image of the banana. First, arrange the base wire frame on the image.


Drag the wire with the mouse and adjust it so that it overlaps the image perfectly.


After adjustment, set the virtual light source.


Shadows are drawn based on the positional relationship with the object.


Render all of them and create images.


However, although it was an image, the arrangement of banana was allowed to move freely.


Yellow cab running in town ......


It proliferated with straw. It looks just like a copype and it is a big difference from normal image processing that the angle has changed properly depending on the place where it is actually arranged.


And, when I thought that it floated, I flew away as it was.


A pen placed on paper starts moving ......



Fruits expressed in paintings also move.



Flights of fighters that fly to the left on the screen ......


You can turn it as if you fly to here.


It is a technology that enables surprising image processing, but its details are described in detail in 13 papers through mathematical expressions and so on, so it may be interesting for those who are interested I do not.

3D Object Manipulation in a Single Photograph using Stock 3D Models - SIGGRAPH 2014.pdf
(PDF file)http://www.cs.cmu.edu/~om3d/papers/SIGGRAPH2014.pdf


Also, since the source code for Mac OS X has been released, it was also possible to actually try it.

OS X Source Code
http://www.cs.cmu.edu/~om3d/code/OM3D_1_0_0_source.zip