Aug 01, 2020 15:00:00

What is the 'content' of a JPEG image?

JPEG is one of the most used image compression formats, but how JPEG compresses images and how computers process JPEG data is less well known than JPEG.

Angealbertini, who is involved in security audits at Google, explains the structure of such JPEG data.

formats/jpeg.md at master · corkami/formats · GitHub
https://github.com/corkami/formats/blob/master/image/jpeg.md

The official name of JPEG is 'Joint Photographic Experts Group', which is one of the image compression formats. Displaying a JPEG with the Linux command 'xxd' that displays a file in hexadecimal binary looks like this. It looks like random strings are lined up, but the binary list makes sense.

JPEG holds 'data type', 'data length', and 'data itself' in segments, and has a simple structure in which these segments are linked. The type of data is represented by a part called Marker, which is defined by a byte 'ff' followed by a non-zero byte. For example, 'ffd8' displayed at the beginning of the binary above means the beginning of image data, and is a Marker called 'SOI'.

Conversely, at the end of the binary, 'ffd9', which is the 'EOI' Marker that means the end of the image data, is displayed.

Angealbertini says that most JPEG file binaries start with the pattern 'FF D8 FF E0 00 10 .J .F .I .F 00'. Next to 'FFD8' which is SOI, there is a reserved area called 'Application 0 (APP0)', which starts from Marker called 'FFE0'. Information such as an identifier such as 'JFIF' and resolution is stored in the APP0 area.

Next to the APP0 area is the “quantization table”, which is represented by the Marker “FFDB”. The JPEG image is divided into blocks of 8 x 8 dots, and the brightness and color difference in each block are expressed as a waveform on the coordinate axis. The details of this waveform are simplified by a calculation called the Discrete Cosine Transform , but the quantization table is used for rounding down the simplified data and further compressing the data. The quantization table allows you to control the image quality, etc. The larger the fraction is rounded down, the coarser the image quality after JPEG conversion will be. The reason why JPEG is lossy compression is because the fractional part of the numerical value is truncated at this time.

There is a 'START OF FRAME (SOF)' after the quantization table, which is represented by a Marker such as 'FFC0'. It is said that the size of images is defined in SOF.

The 'Huffman table' that begins with Marker called 'FFC4' is an area for

Huffman coding to further compress the data. The Huffman code can compress data by classifying the data according to the frequency of occurrence, arranging the data with low frequency of occurrence in the deep part of the binary tree that branches by 0 and 1, and encoding from the beginning of the binary tree. Possible algorithms. In JPEG, there are two stages of compression process: quantization and Huffman coding.

'START OF SCAN' and 'IMAGE DATA' store the component information of the image data and the image data itself.

The overall image of the binary that makes up the JPEG image looks like this. Image software can display JPEG images by reading data with the help of Marker.