Feb 05, 2019 19:00:00

Automatically colorize monochrome photos with deep learning 'ColouriseSG'

The birth of an attempt to utilize government-related technology in Singapore is "Color SG ", a web application that automatically colors monochrome photos. ColouriseSG is a tool created using deep learning to color monochrome photographs taken at the Chinese Girls School in Singapore from 1890 to 1923, and government-related engineers engaged in development worked on blogs I am talking about that secret story inside.

Bringing black and white photos to life using Colorise.sg
https://blog.data.gov.sg/bringing-black-and-white-photos-to-life-using-colourise-sg-435ae5cc5036

"ColouriseSG" which automatically colors monochrome photos is a free web application that you can use from below.

ColouriseSG

It's easy to use, first visit the page and click "Try it yourself".

Click "Select photo".

Then select the monochrome photo you want to color and click "Open".

Then the monochrome photograph and the colored photograph which became the original are displayed at the same time as follows. The deep learning model used by ColoriseSG stands out human subjects and natural landscapes, and using a high resolution image will make it better color. In addition, monochrome photos uploaded for coloring will not be saved on the server.

You can also download pictures actually colored.

A software engineer, Preston Rim, who worked on the development of ColouriseSG said, "Have you ever wondered if you saw old black and white photographs? Was the person who took the picture actually watched the sight? Is it? "I said. Although ColouriseSG is not to accurately reproduce the color tone when monochrome photos were taken, it adds a color that does not give a sense of incompatibility to the photograph, "What kind of sight can be seen at the moment a monochrome photograph was taken It can be helped to imagine what it was. "

The rims seemed to have created ColouriseSG in Hackathon conducted in January 2019, especially coloring monochrome pictures taken at the Chinese Girls School in Singapore between 1890 and 1923 "Deep Learning Color Riser Tool "It is written that it developed as. The coloring point of ColorsizeSG is "to generate images with plausible colors" and it does not mean to accurately reproduce the actual color when black and white photographs were taken. Rim's development team also noted that "coloring black-and-white photographs using deep learning is an active research field, and their models are never perfect."

Rim says, "It is true that there are tools to colorize monochrome photos, but as far as we can tell, there is no tool to train colorization in line with the historical background of Singapore." I will. As for why "coloring in line with the historical background of Singapore" is necessary, even with Algorithmia, which is an existing colorization tool, in the photos taken in the past in Singapore, a coloring with a very high degree of precision It seems to be because it can not be done.

The reason why older monochrome photos in Singapore are not well colored with Algorithmia is simple because the photos used in the Algorithmia training data set and the old black and white photos in Singapore are so different. In Algorithmia, coloring was learned using 1.3 million images from ImageNet, an image database created by researchers at Stanford University and Princeton University. "Image related to Singapore on ImageNet It is unlikely that there will be, "Rim pointed out.

Therefore, Mr. Rim's development team tried to make coloring of old historical photos more credible by creating coloring tools learned more from historical photos unique to Singapore . For coloring photos, the artist color to color human monochrome photos roughly speaks "1: Survey on historical, geographical and cultural background of photographs to derive appropriate colors for black and white photography "2: Coloring black-and-white photos using software tools such as Photoshop" is said to perform two tasks. Therefore, it seems that Rim's development team thought that it is necessary for the computer program to do the following program.

1: Identify the object from monochrome photographs and determine the plausible color of the object from the images that have been seen in the past
2: Color black and white photos

And the rims decided to use hostile generation networks (GANs) for coloring monochrome photos. This GANs is the first neural network to act as a "generator" and has more than 20 million mathematical parameters trying to predict color values of various pixels in a monochrome picture based on features in the image. Furthermore, it seems that the neural network of GANs is utilized also to identify whether the generated color is photorealistic compared with the original photograph.

I will train the first neural network, or generator, until the second neural network judges "coloring is photorealistic". In addition, we use fast.ai and PyTorch which are popular for model development, and use the photo of more than 500,000 old Singapore in the data set. They said they trained using a local GPU cluster equipped with NVIDIA V100 GPU.

At this point, the deep learning model was in the local GPU cluster located in the office where the development team worked, and the deep learning-based color riser tool that we created was only accessible to Rim's development team. However, the development team needed to save the deep learning model on the Internet so that anyone can use the color riser tool. So the rims decided to use the Google cloud platform. Finally it is possible to color monochrome photos uploaded in about 3 seconds.

Rim's development team has named the deep learning-based coloring tool created "ColouriseSG" and is open to the general public. In addition, the development team coloredise (right) monochrome photos (left) provided as materials from the New York Public Library and the National Archives of the Singapore (right), especially blogging examples successfully coloring successfully Published on above.

Picture of Singapore's first president Yusof Bin Isak and Malaysian politician Ismail Abdul Rahman , taken on Malaysia Day on September 16, 1963