Solve "Bongard puzzle" by deep learning which guess classification rules from patterns grouped into two

The process of creating a program to solve a puzzle called Bongard puzzle using deep learning was published on the blog.

Solving Bongard Problems With Deep Learning | Sergii Kharagorgiev
https://k10v.github.io/2018/02/25/Solving-Bongard-problems-with-deep-learning/#perfundo-untarget

Bon Guard PuzzleIs a puzzle that guesses the rules used to classify the left and right groups by observing the patterns divided into six on the right and left as follows. For example, in the image below, you can see that the left is divided into "triangle" and the right is divided into "quadrangle" group.


Another example as an example. What are the elements that divide the left 6 and the right 6 ...?


The answer is "The figure on the left isConvex polygonBut the right is not convex. " The process of making a program to solve such bongard puzzle using deep learning is open to the public. The upper two were a difficulty to understand the answer just by thinking a little, but there are some more difficult problems like the image below.

· Question 1Click here to display the correct answer


· Question 2Click here to display the correct answer


· Question 3Click here to display the correct answer


· Question 4Click here to display the correct answer


According to Sergii Kharagorgiev who published this entry, there are at least two major problems in solving this problem. It is a problem of "There are only 12 images for learning to be used for deep learning, total of 12 images" and "the input is an image but the output needs to describe the classification standard in natural language".

So I will simplify the problem solving in natural language into a classification problem. In the classification problem that we use 10 pieces in total of 5 images out of 6 images originally for training and correctly classify the remaining 2 images without information "originally left or right". I will change it. By doing this you can avoid answers in natural language.


In fact, I used "transitional learning" which reuses parameters learned with other data in actual solving. Also, the image of Bon Guard puzzle is too small to use for learning the neural network, so we need to create a new data set.

Kharagorgiev used the preliminary learning of a neural network which generates an image resembling a figure appearing in a bongard puzzle and extracting a feature amount. The generated image is a basic geometric figure, like the image below. It seems that he has learned by generating a total of 1 million images.


He said that he edited and used the "Darknet Reference" model in his neural network. I am using nine convolution layers.


actuallyThe page where the Bon Guard puzzle is gatheredWhen we solved the 232 pieces of the problems that in, there are 47 that are divided into left and right properly "test" image, of the thing of which 41 pieces were be classified correctly. The following image green part is a problem that was correct in, shows the problem that had the wrong red portion. Part of the white is even two test images will be classified to the left or right side, could not answer the problem.


Kharagorgiev stated that a simple pre-formed deep learning technique was found to be useful for solving the Bon Guard puzzle, and in order to do the original "language explanation" rather than "classification" like this time It is difficult to process natural languages ​​and concludes that it is a benchmark of machine learning which is challenging even 50 years after the Bon Guard puzzle has been made.