Feb 21, 2021 17:00:00

How does a computer recognize the 'human eye'?

'

Computer vision ' is a research field that solves the problem of how computers understand digital images and movies. Engineer Fabrizio Di Guardo explains how computer vision algorithms are used through his own EyeDraw algorithm, which allows you to draw lines by moving your eyes.

EyeDraw — How to draw with your eyes using Computer Vision | by Fabrizio Di Guardo | Jan, 2021 | Towards Data Science
https://towardsdatascience.com/eyedraw-how-to-draw-with-your-eyes-using-computer-vision-226317501e6a

EyeDraw is software for drawing lines from the perspective developed in Python using the open source computer vision library '

OpenCV '. Guardo explains that EyeDraw uses three algorithms to recognize the line of sight: ' face detection by Haar Cascades ', ' detection of eyes as blobs ', and 'mapping of eyes and cursor by homography '. ..

The first step is face and eye recognition. As a recognition method, it is said that image analysis using the feature amount identifier of Haar Cascades using the pre-learning model provided by OpenCV is adopted. The feature amount identifier plays a role of making the computer understand that 'the face and eyes have such features', and by superimposing the identifier on the face image, the computer is made to capture the features of the face and eyes.

After recognizing the face and eyes, Guardo explained that the next step is to detect the eyes as 'blobs'. Blobs are 'pixel groups that share certain properties' for computers, and are suitable for detecting pupils. After converting the image to a numerical value, the pupil is detected by estimating the center of the contour detected using the

openCV findContours method and its radius.

The next step is to associate the detected pupil with the screen cursor. Mr. Guardo says that he uses a graphic processing technology called '

homography (projection transformation) ' to realize this correspondence. Homography is a technique for associating two points on different planes. Homography here is a 3x3 matrix that mathematically transforms a 3D vector representing a point in a 2D coordinate space into a 3D vector representing another point.

In order to make the correspondence between the pupil and the cursor by homography, 'calibration' is required to inform the computer of the 'correct correspondence'. The calibration function is also provided in EyeDraw, and it can be easily realized by using the OpenCV method.

With the process up to this point, the computer can now recognize 'where the user is looking on the screen'. However, this alone does not allow the computer to determine whether the cursor movement is 'drawing a line' or 'moving the cursor to another location', so Guardo added 'pointer mode' and 'cursor mode' functions. I've implemented it and manually switched to fix the problem.

In recent years, deep learning has become a powerful competitor to computer vision. However, Guardo points out the complexity of deep learning and says that theoretically clear computer vision continues to be used in areas such as medicine and security.

'EyeDraw' developed by Guardo is published on GitHub and can be used by anyone for free.

GitHub --fabridigua / EyeDraw: Small computer vision software that allows you to draw with your own eyes
https://github.com/fabridigua/EyeDraw