Sep 14, 2021 11:30:00

Facebook announces proprietary application compression technology 'Superpack'

Facebook announced that it has developed 'Superpack ', a compression technology for Android apps. According to Facebook, Android apps such as Facebook, Instagram, WhatsApp, and Messenger have succeeded in reducing the size by an average of 20% or more compared to

Android's default ZIP compression.

Superpack: Pushing the limits of compression --Facebook Engineering
https://engineering.fb.com/2021/09/13/core-data/superpack/

Developers check a lot of code every day and add it to the app that will eventually be downloaded to their smartphone. If you add this code as it is, the size of the app will grow and eventually the time it takes to download the app will exceed the allowable range.

To minimize the size of your app, you need to compress your data. Compression allows billions of users around the world to download smaller apps faster and have less traffic. These ideas are especially important in areas where mobile phone communication capacity is limited. However, compression alone is not enough to accommodate app updates and feature additions, so Facebook has developed a technology called 'Superpack.'

Superpack combines compiler analysis and data compression to enable size optimization that exceeds the capabilities of conventional compression tools. 'Superpack pushes the boundaries of compression and delivers significantly better compression ratios than existing compression tools,' Facebook said.

Below is a bar graph comparing the size of each application after it is actually compressed with ZIP (light blue), Xz (blue), and Superpack (orange).

According to Facebook, Superpack's strength is its ability to compress

machine code, bytecode, and other structured data. The approach is to 'define the information content of the data as the length of the shortest program that can generate that data, based on the Kolmogorov complexity algorithm, which measures complexity.' Facebook is 'short, can compress the data by expressing as a program for generating data. For example, Fibonacci numbers program that generates can be said to those highly compressed files containing a plurality of Fibonacci numbers '.

Superpack is designed with OCaml on the compression side and C on the decompression side. Therefore, it seems that compression will be slower than decompression.

In addition, Superpack targets structured data, codes, integers, and character string data, and does not target images, videos, and audio files at the time of article creation. At the time of article creation, only Facebook engineers can use Superpack, but in the future we aim to make it public and provide it to everyone, and apply it to other platforms such as iOS or open source it. Facebook said it was also aiming for that.