NVIDIA develops AI technology "Super SloMo" capable of ultra-smooth slow motion of images shot with ordinary cameras

NVIDIA collaborates with Merced University and the University of Massachusetts College Amherst to develop a technology to convert normal movies into smooth slow motion movies by predicting and automatically generating intermediate frames of movies using machine learning "Super SloMo"Was developed.

[1712.00080] Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation
https://arxiv.org/abs/1712.00080

Transforming Standard Video Into Slow Motion with AI - NVIDIA Developer News Center NVIDIA Developer News Center
https://news.developer.nvidia.com/transforming-standard-video-into-slow-motion-with-ai/?ncid=--43539

You can understand the high quality of movies automatically generated by Super SloMo with a single shot if you look at the movie below.

Research at NVIDIA: Transforming Standard Video Into Slow Motion with AI - YouTube


Fierce battle before the goal of ice hockey. Where you want pictures based on slow motion in order to check play accurately.


In the movie, you can view the Super SloMo movie (right), which was made slow motion by creating an intermediate frame from NVIDIA developed AI from the normal shot video (left) and normal shoot video (left), side by side Comparison. You can see that the image that was converted to slow motion with the Super SloMo technology is drawing motion smoothly without collapse.


A scene that a man jumps off to a huge balloon floating in a pool.


Compared with Super SloMo (right) is a slow motion picture taken with a high speed camera.


Next, photographing how to break water balloon with a tennis racket.


It is confirmed that Super SloMo's slow motion video (below), which captures the scattering of water clouds from between the guts, maintains almost the same quality as the slow motion video taken with a real high-speed camera I will.


"Super SloMo" developed by the collaborative research group of NVIDIA · Merced University · Massachusetts College Amherst School is to predict and correct intermediate frames from multiframe rather than single frame correction. In machine learning, we used NVIDIA's "Telsa V100", a deep learning library "cuDNN" and a deep learning framework "Pytorch" and combined 11,000 movies taken regularly at 240 fps and active images of sports I trained and trained, and trained a convolution neural network. After training, I used another data set to further verify system accuracy.

As a result, Super SloMo can create a movie with a high frame rate to 240 fps or 480 fps by automatically generating intermediate frames of movies taken at 30 fps or 60 fps, and a beautiful slow throw played in the above movie I have succeeded in creating motion images.

The wonderful thing about Super SloMo is that it can be changed to a smooth slow motion movie as if you shot a picture taken with a camera at super high speed. With Super SloMo, you can not only create slow motion movies with ordinary camera shooting, but also you can slow motion the already shot images.

About Super SloMo will be held in Salt Lake City, UtahCVPR 2018It will be showcased in.