— Ch. 1 · Origins And Evolution —
TensorFlow.
~4 min read · Ch. 1 of 5
In 2011, the Google Brain team began building a proprietary machine learning system called DistBelief. This early software relied on deep learning neural networks to process vast amounts of data for Alphabet companies. By 2009, Geoffrey Hinton and his colleagues had implemented generalized backpropagation techniques that reduced speech recognition errors by 25 percent. The team assigned multiple computer scientists including Jeff Dean to refactor this codebase into a faster application-grade library. They released the initial version under the Apache License 2.0 in 2015. Version 1.0.0 officially launched on the 11th of February 2017. During the June 2016 Google I/O Conference, Jeff Dean noted that 1,500 repositories on GitHub mentioned TensorFlow with only five coming from Google itself. An updated major release arrived in September 2019 when the team introduced TensorFlow 2.0. This update shifted the automatic differentiation scheme from static computational graphs to Define-by-Run methods popularized by Chainer and PyTorch.
Hardware Acceleration
Google announced its Tensor Processing Unit or TPU in May 2016 as an application-specific integrated circuit built specifically for machine learning workloads. These chips delivered an order of magnitude better optimized performance per watt compared to existing hardware during their first year of operation inside data centers. The second-generation TPUs became available in May 2017 through Google Compute Engine offering up to 180 teraflops of performance. Clusters of 64 second-generation units provided up to 11.5 petaflops of total processing power. Third-generation TPUs arrived in May 2018 delivering 420 teraflops alongside 128 GB high bandwidth memory. Cloud TPU v3 Pods offered over 100 petaflops of performance with 32 TB HBM capacity. In February 2018, Google made these accelerators available in beta on the Google Cloud Platform. July 2018 saw the announcement of the Edge TPU designed to run TensorFlow Lite models on small client devices like smartphones. This purpose-built ASIC chip enabled edge computing capabilities previously unavailable on mobile hardware.