How does Facebook encode hundreds of millions of videos a day?
Hundreds of millions of videos are uploaded daily to Facebook, the world's largest social media. However, to ensure that all videos are delivered at the highest quality and resolution with as little buffering as possible, you need to encode each video with the best codecs and settings. Facebook Engineering, Facebook's engineering department, explains on its official blog about new efforts Facebook is making to optimize video encoding.
How Facebook encodes your videos --Facebook Engineering
When a video is uploaded to Facebook, it will start encoding to multiple resolutions such as 360p, 480p, 720p, 1080p depending on the average bit rate. At this time, the compression efficiency, visual video quality, and required computing power will change depending on the codec and settings used.
Of course, you can't encode a large number of videos at the same time, so there is a priority in the video processing queue, and they are done in order. It seems that this priority was once decided based on various factors such as the total playing time of the video, the content of the video, the number of friends and followers who uploaded the video. Facebook said that it was trying to optimize the user experience by appropriately assigning the priority of video encoding.
However, Facebook admits that this approach had its drawbacks. As new video codecs and associated settings are introduced every day, the video coding patterns are enormous and it is impossible to completely optimize the user experience.
In addition, Facebook points out that 'the pattern of video consumption on Facebook is biased.' For example, if you compare the Facebook page of a large company like Disney with the Facebook page of a typical small and medium-sized video blogger, Disney has fewer video uploads than Vlogger, but the total video play time is overwhelming. It will be longer and will inevitably be a higher priority. However, prioritizing pages that consume a lot of video will ignore the support of small creators.
So Facebook says it has introduced a
Facebook has defined profits, costs and priorities as follows:
-Profit = 'Relative compression efficiency of encoding family with fixed quality' x 'Effective predicted total playback time'
-Cost = Computation cost of missing encoding in family
・Priority = profit ÷ cost
An 'encoding family' is a set of encoding files that can be distributed together. For example, H.264 360p, 480p, 720p, and 1080p can be said to constitute one encoding family. And 'compression efficiency' indicates 'how many minutes will the video play time when encoding a 1GB video with a specific codec and settings'.
'Effective predicted total playback time' is the total playback time of a video predicted by a machine learning model. This shows how many devices the encoded video can be played on. For example, assuming that 20% of devices trying to play a video with a predicted total playback time of 100 hours cannot play a VP9 encoded video, then a widespread H.264 encoded video Even if the effective predicted playback time is 100 hours, it will be about 80 hours when encoded with VP9.
The machine learning model predicts the total play time based on the number of friends and followers of the video uploader, the average total play time of the videos uploaded so far, the metadata of the video, the past popularity on the platform, etc. To do. According to Fcebook, past total playback time data is the best predictor in predicting total video playback time. Furthermore, it seems that the total playback time will change greatly depending on whether it is a story video of only a few seconds or a live distribution of several hours.
By adopting a new priority calculation method using a machine learning model, Facebook has become able to encode uploaded videos much more efficiently than before. 'As a result, it's easier to continue investing in newer and more advanced codecs to provide Facebook users with the highest quality video experience,' Facebook said.
Related Posts:
in Software, Web Service, Posted by log1i_yk