Why is Twitter moving to a machine learning library to TensorFlow?

Twitter uses machine learning to improve the service so that it displays the contents tailored to the user. At the time of writing the article, Twitter is to machine learningTorchAlthough it is using, developed by Google from the summer of 2017TensorFlowWe are making efforts to shift to. Twitter's software engineer Nicolas Leonard and Sibeli Montes Hallas explain the benefits of moving to TensorFlow.

Twitter meets TensorFlow
https://blog.twitter.com/engineering/en_us/topics/insights/2018/twittertensorflow.html

Twitter in 2014Deep learningWe acquired Madbits to tackle image search and database construction using the company, and launched the company "Cortex" dealing with deep learning.

After launching Cortex, machine learning gradually gets permeated in Twitter company, but it was not controlled such as different machine learning library used by the team. So in the mid 2016 Cortex began developing a machine learning platform to be used in common within the company in order to take control of development within Twitter. As a result,LuaTorch based Deep Bird described in language was born, and it seems that it made a big effect on improvement of service etc.


However, in 2017, Torch's development community began developing languagesPythonBy moving to this, Torch support for the Lua language will be gone. So, Cortex is Python version of TorchPyTorchEvaluate whether to move to TensorFlow. As a result of numerous discussions, Cortex decided to move to TensorFlow.

The main reason why Cortex came to adopt TensorFlow is developer support provided by TensorFlow. TensorFlow includes a distributed processing framework for large-scale dataApache HadoopFile system "HDFSIt supports a large development community and many documents. In addition to this, there is TensorBoard which can be used as a debugging tool, which makes it possible to increase development efficiency.


Based on these, Cortex which adopted TensorFlow started development of DeepBird 2 based on TensorFlow. Naturally, development of Deepbird 2 is not smooth, and it has come to confront various problems such as deterioration of processing performance due to problems caused by differences in implementation between Torch and TensorFlow.

Somehow, Cortex which solved all these problems said that as of June 2018 document creation is done in order to popularize DeepBird 2 to in-house engineers. Deepbird 2 is expected to be applied to Twitter services in the next few months, and we are striving to improve timeline display etc after application.