``CoDeF'' to improve the drawback of ``screen flickering'' that is common in AI-generated videos
When watching AI-generated videos, objects suddenly change into something else in a continuous motion, ignoring context, or the image jerks and shakes like a ``sketchy flipbook''. I have things to do. ' Content Deformation Field (CoDeF: Content Deformation Field) ' has been released to improve such problems.
CoDeF
[2308.07926] CoDeF: Content Deformation Fields for Temporally Consistent Video Processing
https://arxiv.org/abs/2308.07926
You can check the demo video of CoDeF from the following. Processing video with CoDeF allows you to add dramatization to the input video in a natural way.
Demo video of 'CoDeF' Part 1 - YouTube
Below, the video on the left is the input video, and the video on the right is the output video. The figure of a woman has changed to Tifa Lockhart appearing in 'Final Fantasy'.
Demo video of 'CoDeF' Part 2 - YouTube
Below is a video of the train running through the grassland turning into an anime style.
Demo video of 'CoDeF' Part 3 - YouTube
CoDeF consists of a 'canonical content field' that aggregates the static content of the entire video, and a 'canonical image' that is recorded when transforming the image 'canonical image' rendered from the canonical content field into frames along the time axis. It is composed of 'time deformation field', and it is possible to create 'image with frame consistency' based on the input image.
The CoDeF repository is published on the following page.
qiuyu96/CoDeF: Official PyTorch implementation of CoDeF: Content Deformation Fields for Temporally Consistent Video Processing
https://github.com/qiuyu96/CoDeF
Related Posts:
