Google AI succeeds in generating an intermediate frame from 'the first and last frame of the movie'
According to the dissertation published by the AI research team at Google on the public site
From Here to There: Video InbetweenUsing Direct 3D Convolutions
(PDF file) https://arxiv.org/pdf/1905.10240.pdf
Google's AI can create videos from start and end frames alone |
https://venturebeat.com/2019/05/28/googles-ai-can-create-videos-from-start-and-end-frames-alone/
Google research researchers Yunpeng Li, Dominik Doblek, and Marco Tagliasacchi who are researching AI have developed an AI that generates plausible movie pieces from the beginning and end of the movie. .
The developed AI consists of a deep neural network based on the visual cortex of animals, and consists of elements such as a 2D convolutional image decoder, a 3D convolutional latent expression generator, and a video generator. First, the image decoder maps the first and last frames into the potential space, the latent expression generator learns the information of the input frame, and the video generator finally generates the middle frames.
by
The researchers used three sample movies to test the performance of the developed AI. Each sample was a short movie of 16 frames, and the resolution per frame was reduced to 64 x 64 pixels. Only the first and last frames extracted from this movie were given to AI, and 14 frames were generated along the way.
AI ran the model 100 times for each sample movie, modified the model and repeated the same process about 10 times. The experiment was conducted using data GPU Nvidia Tesla V100 , and it took about 5 days in total.
The following images are the three movies created as a result of the experiment, comparing the frames created by AI with the actual frames. For each movie, the top row is the frame generated by AI, and the bottom row is the frame extracted from the actual movie. It seems that there is not much difference between the frames generated by AI and the actual frames.
The researchers say, 'Imagine a system that can automatically create animation from manga pieces. It will definitely revolutionize the animation industry.' At the time of writing, AI did not have the ability to create beautiful animations from manga pieces, and the technology did not catch up, but the progress of AI made it a concrete goal to save labor in the animation industry.
Related Posts:
