TensorFlow

• 1. Introduction

  ›

  TensorFlow is a software library, and by June 2016 it had already spread far beyond the company that built it. At the Google I/O Conference that month, engineer Jeff Dean shared a striking number. Of the 1,500 GitHub repositories that referenced TensorFlow, only five belonged to Google itself. A tool built to speed up Google's internal research had quietly become something much bigger. This documentary traces how that shift happened. It follows a research library that ended up training software used in hospitals, social media feeds, and classrooms far from Google's own labs.

• 2. From DistBelief to TensorFlow

  ›

  In 2011, Google Brain built DistBelief, a proprietary machine learning system based on deep learning neural networks. Its use grew fast across many different companies inside Alphabet, both for research and for commercial products.

  In 2009, a team led by Geoffrey Hinton had already implemented generalized backpropagation and other improvements to these networks. The changes allowed for neural networks with substantially higher accuracy, including a 25 percent reduction in speech recognition errors.

  Jeff Dean was one of several computer scientists Google assigned to rebuild DistBelief into a faster, more robust, application grade library. That rebuild became TensorFlow, described as Google Brain's second generation system.

  Google first released TensorFlow under the Apache License 2.0 in 2015, making it free and open source software from the start. Version 1.0.0 followed on the 11th of February 2017. Its reference implementation runs on a single device, but TensorFlow can also spread across multiple CPUs and GPUs. Optional CUDA and SYCL extensions add support for general purpose computing on graphics processing units.

  TensorFlow runs on 64 bit Linux, macOS, and Windows, along with mobile platforms such as Android and iOS. Its flexible architecture lets computation move across CPUs, GPUs, and TPUs. That means scaling from a single desktop to clusters of servers, or down to mobile and edge devices.

  The name TensorFlow comes from tensors, the multidimensional data arrays that its neural networks operate on. Every computation in the system is expressed as a stateful dataflow graph, tracking how data moves and changes as it flows through the network.

  That flexible architecture, built to run across CPUs, GPUs, and TPUs, set the stage for hardware Google would soon build to match it.

• 3. Built for Every Chip

  ›

  In May 2016, Google announced its Tensor Processing Unit, or TPU. The chip was an application specific integrated circuit built specifically for machine learning and tailored to TensorFlow. Rather than training models, the chip was designed to run them, delivering high throughput of low precision arithmetic such as 8 bit calculations. Google said it had already been running TPUs inside its own data centers for more than a year. The company reported an order of magnitude better performance per watt for machine learning work.

  By May 2017, Google had a second generation TPU ready, along with access to it through Google Compute Engine. These chips delivered up to 180 teraflops of performance on their own, and when grouped into clusters of 64, reached up to 11.5 petaflops.

  A year later, in May 2018, Google's third generation TPUs pushed performance up to 420 teraflops, backed by 128 gigabytes of high bandwidth memory. Cloud TPU v3 Pods combined these chips into clusters offering more than 100 petaflops of performance and 32 terabytes of that same high bandwidth memory.

  In February 2018, Google made TPUs available in beta on the Google Cloud Platform, opening the hardware to more outside developers.

  Five months later, in July 2018, the company announced Edge TPU. This was a purpose built ASIC chip designed to run TensorFlow Lite models on small client devices such as smartphones known as edge computing.

  In May 2017, Google announced TensorFlow Lite, a software stack meant for mobile and embedded devices. A developer preview followed in November 2017, giving outside programmers their first hands on look.

  In January 2019, the TensorFlow team released a developer preview of a mobile GPU inference engine. It used OpenGL ES 3.1 Compute Shaders on Android devices and Metal Compute Shaders on iOS devices.

  In May 2019, Google announced that TensorFlow Lite Micro, also called TensorFlow Lite for Microcontrollers, would merge with ARM's uTensor project. The combined line was later renamed LiteRT in 2024.

  That renaming marked how far the mobile side of TensorFlow had traveled since 2016, even as the core library itself was about to change shape entirely.

• 4. The Shift to 2.0

  ›

  TensorFlow's share of research papers was slipping as PyTorch gained ground, and Google answered with a new major version in January 2019. That version, TensorFlow 2.0, became officially available in September 2019.

  TensorFlow eager was the most significant change inside version 2.0. It replaced the old static computational graph with a Define-by-Run scheme, first popularized by Chainer and later adopted by PyTorch.

  Eager execution meant operations were evaluated immediately as they were written, rather than added to a graph to run later. Programmers could step through the code line by line in a debugger, since data changed at each line instead of somewhere inside a hidden graph.

  TensorFlow 2.0 also cleared out older libraries that had accumulated in the framework, and added cross compatibility so models trained on different TensorFlow versions could work together. GPU performance improved significantly as part of the same release.

  Those version 2.0 changes reshaped the tools built directly into TensorFlow's core, tools that go well beyond simply running a neural network.

• 5. Inside the Toolkit

  ›

  Backpropagation depends on gradients, and TensorFlow's autodifferentiation feature calculates that gradient vector automatically for every parameter in a model. To do this, the framework tracks the order of operations applied to input tensors, then computes gradients for the right parameters afterward.

  TensorFlow's distribute API spreads computation across multiple devices using a choice of distribution strategies, in both eager and graph execution. This kind of distributed computing speeds up training and evaluation and has become common practice across the field of AI.

  Mean squared error and binary cross entropy are two of the loss functions, also called cost functions, that TensorFlow provides to train and assess models.

  Precision, Recall, and Intersection over Union sit alongside binary, categorical, and sparse categorical accuracy among the metrics TensorFlow exposes for judging a model's performance.

  TensorFlow.nn handles primitive neural network operations, including several kinds of convolutions such as 1D, 2D, 3D, Atrous, and depthwise versions. It also covers activation functions like Softmax, RELU, GELU, and Sigmoid, along with operations such as max pooling and bias add.

  ADAM, ADAGRAD, and Stochastic Gradient Descent are among the optimizers TensorFlow offers for training neural networks. Each one tunes a model's parameters differently, which can change how well and how quickly that model converges.

  Every one of these tools is reachable well beyond TensorFlow's original Python home.

• 6. Many Languages, Many Tools

  ›

  Keras sits at the center of TensorFlow's own APIs, giving users a way to build their own machine learning models. Beyond building and training a model, TensorFlow can also load the data used to train it and deploy the finished model through TensorFlow Serving.

  Python is the only language with a stable Application Program Interface in TensorFlow, while JavaScript, C++, and Java get APIs without a backwards compatibility guarantee. Third party developers have built further bindings for C#, Haskell, Julia, MATLAB, Object Pascal, R, Scala, Rust, OCaml, and Crystal.

  Go and Swift once had their own TensorFlow bindings too, but both are now archived and no longer supported.

  In March 2018, Google announced version 1.0 of TensorFlow.js, bringing machine learning to JavaScript. The library lets users run TensorFlow.js models or models converted from TensorFlow or TFLite, retrain those models, and run everything directly on the web.

  FlatBuffers, rather than the Protocol Buffers format used by standard TensorFlow models, is the data serialization format behind LiteRT. Models built for LiteRT are compressed and optimized to run efficiently on devices with less processing capacity, from phones to embedded chips. Purpose built APIs let mobile apps and embedded devices generate and deploy these models directly.

  TFX, short for TensorFlow Extended, bundles the components needed to run a model through an entire production pipeline. That includes loading, validating, and transforming data, then tuning, training, and evaluating the model before pushing it into production.

  Beyond these core deployment tools sits a wider set of add on libraries built to extend what TensorFlow alone can do.

• 7. The Extended Family

  ›

  Numpy NDarrays are the native datatype of one of Python's most used data libraries. They convert automatically into TensorFlow Tensors whenever they meet a TensorFlow operation, and the reverse holds true as well. That two way conversion means the two libraries work together without programmers writing explicit conversion code. TensorFlow Tensors can even share the same underlying memory as Numpy NDarrays whenever possible, saving on memory overhead.

  TensorFlow Recommenders and TensorFlow Graphics are two of the library's purpose built extensions. Google announced the latter, TensorFlow Graphics, in May 2019 for deep learning in computer graphics. Other add ons include TensorFlow Model Optimization, TensorFlow Probability, TensorFlow Quantum, and TensorFlow Decision Forests, each aimed at a different corner of machine learning work.

  Google Colab, officially called Collaboratory, is a TensorFlow Jupyter notebook environment that needs no setup at all. It runs on Google Cloud, gives users free access to GPUs, and lets them store and share notebooks through Google Drive.

  Google JAX is a separate machine learning framework built for transforming numerical functions. It combines a modified version of autograd with TensorFlow's XLA, or Accelerated Linear Algebra, and follows NumPy's structure and workflow as closely as possible. It also works with both TensorFlow and other frameworks such as PyTorch.

  Four primary functions define JAX: grad for automatic differentiation, jit for compilation, vmap for auto vectorization, and pmap for SPMD programming.

  Every tool described so far works inside code. What TensorFlow actually changed once it left the lab is a different story entirely.

• 8. Applications in the Real World

  ›

  GE Healthcare used TensorFlow to speed up MRI scans and sharpen their accuracy at identifying specific body parts. Google built on the same library to create DermAssist, a free mobile app that lets people photograph their skin to check for potential health problems. Sinovation Ventures took a different direction, using TensorFlow to identify and classify eye diseases from optical coherence tomography scans.

  Twitter used TensorFlow to rank tweets by importance for each individual user, replacing a feed that had simply run in reverse chronological order. VSCO, the photo sharing app, used TensorFlow to suggest custom filters for its users' pictures.

  On the 26th of October 2015, Google officially released RankBrain, a search ranking system backed by TensorFlow.

  InSpace, a virtual learning platform, used TensorFlow to filter toxic chat messages out of classroom conversations. Liulishuo, an online English learning platform, used the library to build an adaptive curriculum that assessed each student's current abilities and chose what material to show them next.

  Carousell, an e-commerce platform, used TensorFlow to generate personalized recommendations for its customers. ModiFace, a cosmetics company, used the same library to build an augmented reality experience letting customers test different shades of make-up on their own face.

  TensorFlow also underlies DeepDream, an automated image captioning tool built on the same library. Research projects like DeepDream suggest TensorFlow's next application has yet to be decided.