What this paper is about
TensorFlow is presented as a machine learning system that operates at large scale and in heterogeneous environments.[S1] The paper describes TensorFlow as both an interface for expressing machine learning algorithms and an implementation for executing those algorithms.[S4] The computation model uses dataflow graphs to represent computation, shared state, and the operations that mutate that state.[S1] The system maps nodes of a dataflow graph across many machines in a cluster and also maps computation within a machine across multiple computational devices.[S1] The devices named in the paper include multicore CPUs, general-purpose GPUs, and custom designed ASICs known as Tensor Processing Units (TPUs).[S1]
The paper states that a computation expressed using TensorFlow can be executed with little or no change across a wide variety of heterogeneous systems, ranging from mobile devices such as phones and tablets up to large-scale distributed systems of hundreds of machines and thousands of computational devices such as GPU cards.[S4] The paper contrasts TensorFlow with previous “parameter server” designs by stating that, in those designs, the management of shared state is built into the system.[S1] The paper states that TensorFlow’s architecture gives flexibility to the application developer by enabling developers to experiment with novel optimizations and training algorithms.[S1]
The paper reports that TensorFlow supports a variety of applications, with particularly strong support for training and inference on deep neural networks.[S1] The paper reports that the system is flexible and can be used to express a wide variety of algorithms, including training and inference algorithms for deep neural network models.[S4] The paper states that TensorFlow has been used both for conducting research and for deploying machine learning systems into production across more than a dozen areas.[S4] The areas listed include speech recognition, computer vision, robotics, information retrieval, natural language processing, and geographic information extraction.[S4]
Core claims to remember
TensorFlow is described as a system that operates at large scale and in heterogeneous environments.[S1] TensorFlow represents computation, shared state, and state mutation using dataflow graphs.[S1] TensorFlow maps nodes of a dataflow graph across many machines in a cluster and within a machine across multiple computational devices.[S1] The paper names multicore CPUs, general-purpose GPUs, and custom designed ASICs known as TPUs as devices that TensorFlow maps computation across.[S1]
TensorFlow is described as an interface for expressing machine learning algorithms and an implementation for executing them.[S4] The paper states that a TensorFlow computation can be executed with little or no change on heterogeneous systems that range from phones and tablets to distributed systems of hundreds of machines and thousands of computational devices such as GPU cards.[S4] The paper contrasts TensorFlow with prior “parameter server” designs by stating that those designs build shared-state management into the system, and the paper states that TensorFlow enables developers to experiment with novel optimizations and training algorithms.[S1]
The paper reports particularly strong support for training and inference on deep neural networks.[S1] The paper reports that TensorFlow is flexible and can be used to express a wide variety of algorithms, including training and inference algorithms for deep neural network models.[S4] The paper reports that TensorFlow has been used for conducting research and for deploying machine learning systems into production across more than a dozen areas, including speech recognition and natural language processing.[S4]
Limitations and caveats
The paper characterizes TensorFlow as operating “at large scale” and in “heterogeneous environments,” and those phrases define the deployment context described in the paper.[S1] The paper’s computation model description is stated in terms of dataflow graphs representing computation, shared state, and the operations that mutate that state.[S1] The paper’s execution description is stated in terms of mapping nodes across many machines in a cluster and mapping computation within a machine across multiple computational devices.[S1]
The paper’s device examples for within-machine execution include multicore CPUs, general-purpose GPUs, and custom designed ASICs known as TPUs.[S1] The paper’s comparison point for architectural flexibility is previous “parameter server” designs where the management of shared state is built into the system.[S1] The paper’s portability statement is phrased as executing a computation expressed using TensorFlow with little or no change across heterogeneous systems from phones and tablets to large-scale distributed systems with hundreds of machines and thousands of computational devices such as GPU cards.[S4]
How to apply this in study or projects
Review the paper’s statement that TensorFlow uses dataflow graphs to represent computation, shared state, and the operations that mutate that state, and restate that model in your own words while preserving the terms used in the paper.[S1] Trace the paper’s description of how TensorFlow maps nodes of a dataflow graph across many machines in a cluster and within a machine across multiple computational devices, and summarize the two mapping levels as they are written.[S1]
Extract the device categories named in the paper and record them as the examples given for multiple computational devices within a machine.[S1] Compare the paper’s description of previous “parameter server” designs, where the management of shared state is built into the system, to the paper’s statement that TensorFlow enables developers to experiment with novel optimizations and training algorithms.[S1]
List the application areas named in the paper for research and production use, and map each area to the paper’s claim that TensorFlow has been used across more than a dozen areas.[S4] Re-read the paper’s portability statement about executing a TensorFlow computation with little or no change from mobile devices to large distributed systems, and quote the examples of system scale that the paper provides.[S4]