Dec 08, 2019 15:00:00

What is the “Exif Orientation attribute” that is a pitfall of image recognition systems?

Normally, image data taken with a digital camera is embedded with metadata such as

Exif (Exchangeable image file) information such as the camera model, shooting conditions, and image orientation. Exif is “a hindrance to smooth image processing,” explains machine learning engineer Adam Geitgey.

The dumb reason your fancy Computer Vision app isn't working: Exif Orientation
https://medium.com/@ageitgey/the-dumb-reason-your-fancy-computer-vision-app-isnt-working-exif-orientation-73166c7d39da

When you shoot with a digital camera or smartphone, information about the orientation of the photo, whether the camera is held vertically or horizontally, is embedded in the image as Exif. For example, if you read a goose photo (left) taken with a smartphone vertically as shown below in Photoshop (right), it will be displayed in the correct orientation.

When you browse Exif of a goose photo, it looks like the following image. If you look at the “Orientation” attribute that specifies the orientation of the image, it says “Rotate 90 CW (rotate 90 degrees clockwise)”. When this part is read by the image viewer, the photo is displayed in the correct orientation.

Exif is a format standardized in 1995 by the

Japan Electronic Industry Development Association with reference to the TIFF format. The JPEG format has been retrofitted, so the old image viewer displays the image itself. Although there is something that can be done, there are things that do not refer to the Exif embedded in the JPEG file.

Image recognition is very popular among machine learning libraries, and the Python face recognition library is one of the most popular machine learning libraries on GitHub . However, these image recognition libraries have been developed as a tool for professionals who manipulate data arrays, and are not intended for general-user-level uses such as “I want to adjust the orientation of images automatically”. As a result, Geitgey explained that many image recognition libraries do not refer to Exif as well as the old image viewer.

Geitgey pointed out, 'If you do not read the correct orientation of the photo, you can not correctly recognize what is reflected in the photo,' 'Exif processing problems are not limited to Python scripts written by beginners and students No, even Google's Google Vision API doesn't handle the Exif Orientation attribute correctly. '

For example, the following is a goose photo read by Google Vision. Since Google Vision does not process the orientation attribute of Exif included in the photo, the portrait photo is displayed in landscape orientation and image recognition is performed as it is. The three geese shown in the photo are detected as 'Animal' and 'Bird' respectively.

When Google Vision is loaded in the correct orientation, everything is correctly detected as “Goose”.

“Comments may be posted on GitHub that the open source library you are using is broken or inaccurate. However, the cause is“ loading the image displayed sideways or upside down. ” 'It's often a simple thing,' replied Geitgey.

Geitgey said, 'If you to read the image on the program written in Python, check the Orientation attribute of Exif, if necessary should rotate the orientation of the image,' said, NumPy array the image in the correct orientation to The following code to load is released on GitHub as a package management tool.