Players who made their own self-made mechanism to record spratoon's opponent's winning percentage appeared

ByAround4o'clock

Nintendo Wii U's game software "Splatoon"Because one game is completed in about 3 to 5 minutes, it is possible to play in a short time in short time, but the function to check the total battle history and winning percentage is not attached," How much winning do you win? " Are you giving up? "You can not know. In the meantime, "I thought that I wanted to know how much my winning percentage was, so I tried to create a mechanism to automatically record victory or defeat"Raspberry PiWith the camera moduleOpenCVPeople who made winning percentage calculation system by oneself are using it.

Raspberry Pi and OpenCV made a mechanism to automatically record Splatoon's victory - feel unreasonable
http://hogesuke.hateblo.jp/entry/ikashot

As a mechanism, first capture game screen using Raspberry Pi's camera module "picamera". I fixed the camera on the cardboard box as below and turned the camera in the direction of the display and took a picture. The author, hogesuke, mistakenly purchased the old model Raspberry Pi, "It takes 6 hours to build OpenCV, 2 hours to build Python, with a specification of CPU 1 core, memory 512 MB, about capturing with picamera There are only about 5 fps coming out, it seems to be pretty challenging. "


Next, it is determined whether or not the character "WIN" or "LOSE" is displayed in the captured image as "OpenCVJudge with. OpenCV is a C language library for image processing and recognition developed by Intel and is provided as open source. The creation procedure is published so that "The procedure until it is possible to recognize the image is roughly as follows".

· Acquisition of sample images
First, three types of samples are prepared: an image (positive sample) showing either "WIN" or "LOSE" or an image not displaying either (negative sample). This time it was 400 samples of WIN sample, 300 LOSE sample and 300 samples of negative sample. It seems that samples gathered at the end of 700 games and about 35 hours of game play. "To improve the accuracy of detection, it is necessary to prepare samples in thousands, but we have compromised because it is painful to do so."

Among the following game screens, the left is the WIN sample, the center is the LOSE sample, and the right is the image of the negative sample.


Feature extraction
Next, we will show the coordinates of the place where "WIN" and "LOSE" on the result screen are displayed, but this work includesTrainingAssistantIt seems that the classification of the sample image and the feature detection were done on the browser using the tool named. Since "WIN" or "LOSE" characters are displayed in the part surrounded by the red frame in the upper left of the image, we will designate this place with all the images.


· Detector generation
Based on a text file showing the generated positive and negative classifications and feature points, detectors are generated using the OpenCV command. This process took the longest time, it seems that it took two full days to complete work on Core i 5, 16 GB of MacBook Pro. It may be due to too many negative samples compared to the positive sample as a cause.

Implementation of OpenCV image recognition code
Finally, we implemented image recognition code using detector. Based on the sample code on the net, it seems that it was easy to implement based on the face recognition code. Looking at the following three images, the image written as "WIN" on the upper left is recognized as a batch, and the blue frame is displayed during recognition. However, according to the author, "Although it detects for a while, the accuracy is bad, it seems that there is also a fatal defect that it is hard to recognize that the room is dark, or wrong WIN as LOSE".


The result of the detection is that by uploading the captured image of the result screen and the result of winning and losing through WebAPI, it automatically records on the web and calculates winning percentage. The outcome of the victory can be seen on the following page.

Squid shot
https://ikashot.net/


The source code isPublished on GitHubBecause it is done, anyone can use it freely. In addition, "To compensate for poor precision, actual code embeds workaround like uploading images as correct detection if more than 3 frames are recognized in 20 frames.The reason for poor precision is that the number of positive samples is I think that it is a small thing, so I think if you collect samples for a while, I will make a detector again. "