A system that fully automatically creates 'film comics' that are manga versions of movies is under research.



'Film comics ' are made by cutting out a large number of still images from video works such as movies and animations, arranging them like manga frames, and adding lines and sound effects. Researchers at the Dalian Institute of Science and Technology in China have developed a system that can create such film comics fully automatically without an editor.

[2101.11111] Automatic Comic Generation with Stylistic Multi-page Layouts and Emotion-driven Text Balloon Generation
https://arxiv.org/abs/2101.11111

A system that automatically generates comic books from movies and other videos
https://techxplore.com/news/2021-02-automatically-comic-movies-videos.html

The researchers' system extracts keyframes from videos, converts them into comic-style images, divides them into frames over multiple pages, and arranges the images in chronological order.



The lines of each character are extracted from the subtitle file included in the audio and video and written in the balloon.



In addition, the size of the characters and the shape of the speech bubbles are changed according to the system's evaluation of the character's emotions by analyzing the voice.



Furthermore, it is also possible to capture the actor's mouth reflected on the screen with a face recognition system and judge 'who is speaking now'.



The following image is a manga automatically created from 4 movies. The works used for the generation are (a) and (e) the 1997 movie ' Titanic ', (b) and (f) the Chinese movie ' The Message ', (c) and ( g) is the drama ' Friends ', and (d) and (h) are the movies ' Up in the Air ' released in 2009.



According to the research team, the selection of keyframes for videos is still inaccurate, and in some cases similar keyframes are selected, which can make the generated manga redundant. Still, the research team said, 'Our experiments show that we can synthesize more expressive and attractive manga compared to previous systems.'

Related Posts:

in Software,   Manga, Posted by log1i_yk