Jul 26, 2021 19:00:00

Will image generation AI change the way art works?

'VQGAN ' is a new image generation model that evolved 'VQ-VAE ', which is a VAE using vector quantization (VQ), into a hostile generative network (GAN). By combining this VQGAN with 'CLIP ', a neural network that connects different data of text and images , Ryan Moulton has created a 'model that generates images from input text'.

Tour of the Sacred Library – Ryan Moulton's Articles
https://moultano.wordpress.com/2021/07/20/tour-of-the-sacred-library/

Mr. Moulton created a 'model that generates an image from the input text' by combining the image generation model 'VQGAN' and the neural network 'CLIP' that connects the image and the text.

Using this model, using the name of illustrator James Gurney , known for his expedition 'Dinotopia, ' set in a fictional island, enter 'James Gurney's Holy Library Hall.' The following image was generated.

Moulton said, 'I found that I could generate amazing images by typing'James Gurney'. This is a great technique. James Gurney is literally an illustrator who'draws the fictional world realistically'. The model I created seems to have learned his style from the online illustrations of James Gurney, and when I instruct him to create one drawn by James Gurney, the best images are generated. I found out. '

In addition, Mr. Morton wrote that 'AI models were not trained to reproduce James Gurney's style', and the data set for model learning is not limited to James Gurney's work.

Below are a number of images generated by Morton with the addition of the text 'James Gurney'. A library with a uniquely shaped ceiling.

The boundaries between the passages and stairs are ambiguous, creating a unique atmosphere.

The library seen from the outside.

A place that seems to be the courtyard of the library.

Is it a passage or a staircase?

It's like a book printing press.

Domed library.

Books are lined up in a cave-like place.

A stream that changes from the library and flows through the gaps between the trees.

There is also an image that expresses something magnificent and deserves to be called art.

Morton said, 'I was scared after doing various outputs using the model. The output image is not' too good '. It is output with the same image generation model that has appeared so far. It's not the 'most impressive' of the images I've created, which means that the optimization process allows for higher quality output. It has the drawbacks of 'inconsistency in space' and 'inability to output human faces', but apart from these drawbacks, the output image is of sufficient quality to fool most casual art viewers. It can be said that it has, 'he said, warning that the overall level of image generation using AI models has improved too much.

In fact, the following is an image automatically generated in the same way with a tool different from the model using VQGAN and CLIP created by Mr. Morton this time. The image automatically generated by Mr. Morton this time was based on a place called 'library', but the following image has nothing to do with the library. As Morton says, it's easy to see that you don't have to have a VQGAN and CLIP combined model to produce great quality images.

Based on these, Mr. Morton said, 'I feel that there is a possibility that the trial and error that AI has done to play chess and Go can be extended to fields such as art and music.' With the advent of AI that automatically generates art works, such as Go AI's 'AlphaGo ', which defeated the world champion of Go, and chess AI , which changed the way chess is played by defeating humans, the way art should be. I am afraid that it may change completely.