Nov 02, 2022 21:15:00

A valuable record of interviewing a famous illustrator who was actually made into an AI model by ``Dream Booth'' that can reproduce painting styles and patterns with image generation AI and the creator of the AI model himself

Image generation AIs such as

Stable Diffusion and DALL E2 use large image datasets for training neural networks. However, there are many cases where the images included in these datasets do not clear the copyright issue, and there are also voices of concern. Meanwhile, an interview with an illustrator who said that his illustration was used for additional learning of the AI model without permission and the person who performed additional learning of the model has been released.

Invasive Diffusion: How one unwilling illustrator found herself turned into an AI model - Waxy.org
https://waxy.org/2022/11/invasive-diffusion-how-one-unwilling-illustrator-found-herself-turned-into-an-ai-model/

Hollie Mengert , a professional illustrator who designs characters for various works, was used for additional learning without permission. One day, Mr. Mengert received an email from a friend or fan with a link to the following thread on the online bulletin board site Reddit.


In this thread, a user named MysteryInc152 uses Mengert's 32 illustrations and uses ' Dream Booth ', which allows additional learning to existing AI models, to create an AI model that can reproduce Mengert's style and patterns. However, MysteryInc152's post triggered a big discussion on Reddit. The thread in question also includes a comment saying, 'How would Mengert himself feel if thousands of people were able to copy Mengert's style exactly, legally or not?'


So software engineer Andy Baio actually made an appointment with Mengert and interviewed him. Mengert said, ``When I first heard this story, I was suspicious that my name was on this AI. I couldn't have said it,' he said.

In the first place, even if Mr. Mengert wanted to allow the use of all images, Mr. Mengert was in a situation where he could not allow it. The images used for AI's additional learning were produced for Disney and Penguin Random House , and Mr. Mengert did not have the rights to the images. Therefore, even Mengert himself could not make a decision as to whether additional learning could be done with Mr. Mengert's images.

Mr. Mengert thought that the illustrations generated by the AI model did not express the individuality of his work. ,that's not necessarily why an illustrator or designer gets a job.Brushstrokes and colors are the most superficial parts of art.What people ultimately resonate with in art is the love, the familiarity. I think it's an easy character, and I'm seeing the AI struggle with that.'

Mengert also said, ``I personally didn't think that AI was making the decision to draw like this, and I felt a gap between the generated results.I am imitating my style. I felt like my name was part of the tool even though I wasn't, and that was frustrating.' I feel like I was thinking something, because if anyone thinks that my illustrations are a reflection of my life and experiences, I don't think they're doing this. What means the most to me is that my name is used, because as a creator I am very happy that my name is used as a stylized image. , to have my name attached to me in this way is ultimately very offensive and invasive to me.'

Meanwhile, Mr. Baio also contacted MysteryInc152, which created the AI model. The identity of Mystery Inc 152 is Mr. Ogbogu Kalu, an engineer from Nigeria and living in Canada. He likes fantasy novels, soccer, manga and anime, and is also interested in image generation AI technology.

Actually, Mr. Kalu seems not to be familiar with Mr. Mengert's work. Originally, there was another user who used Mengert's image and Dream Booth on Reddit for additional learning, but the user's work was not going well, so Kalu improved and adjusted the image set. and released it again.

In a discussion on Reddit, Kalu said, ``There can be no moral argument. You either use Stable Diffusion to contribute to the destruction of today's industry, or you don't.Those who think they can use Stable Diffusion but end up being the 'good guy' because of the silly imaginary lines they draw don't consider themselves. It's deceiving. There is no functional difference.'

When Mr. Baio asked Mr. Kalu about this opinion, Mr. Kalu said he gave a very realistic view. Mr. Kalu believes that using illustrations for learning is legal and likely to be legally judged as fair use, and that it is not possible to protect the pattern with copyright. Also, Stable Diffusion can faithfully reproduce the style of photos and paintings, but the original image is not saved in Stable Diffusion, and although the data set exceeds 100 TB, the AI model data itself It also claimed that 4GB was enough.

However, Baio said, ``If it's fair use, it doesn't matter from a legal point of view what the artist thinks. But when artists don't like this AI, how will their work be used? Do you think we should have a say in what?” Kalu thought for a moment, then said, “Yeah, it’s not. It's a bit different from using directly to replace them.'

Kalu said, 'Many of the people voicing dissent in this case mistakenly believe that they are patching the work together like a collage, but they are creating a new image. It is the past. It's transformative, like trying to recall the vivid memories of the world,' he said. 'Personally, I think this is the point of change. I don't think they really have a say in how they are created.'

After that, when Mr. Kalu learned that Mr. Mengert was not happy about AI, Mr. Kalu changed the name of the model that had additionally learned from Mr. Mengert's work from ``hollie-mengert-artstyle'' to ``Illustration-Diffusion'' and changed it to ``Mengert He is not involved in this AI at all,' he added to the ReadMe.