Tensor Processing Unit

• 1. Origins And Development History

  ›

  In 2013, Google recruited Dr. Amir Salek to establish custom silicon development capabilities for the company's datacenters. Norman P. Jouppi served as the tech lead and principal architect for Google's Tensor Processing Unit development. He led the rapid design, verification, and deployment of the first TPU to production in just 15 months. Three separate groups at Google were developing AI accelerators, with the TPU, a systolic array, being the design that was ultimately selected according to Jonathan Ross. The chip was announced in May 2016 at the Google I/O conference when the company said it had been used inside their data centers for over a year. Google began using TPUs internally in 2015 before making them available for third-party use in 2018.

  51348

• 2. Architectural Design And Performance

  ›

  The first-generation TPU is an 8-bit matrix multiplication engine driven with CISC instructions by the host processor across a PCIe 3.0 bus. It has 28 MiB of on-chip memory and 4 MiB of 32-bit accumulators taking results from a 256x256 systolic array of 8-bit multipliers. A single TPU package contains 8 GiB of dual-channel 2133 MHz DDR3 SDRAM offering 34 GB/s of bandwidth. Norman Jouppi demonstrated that the TPU achieved 15, 30 times higher performance and 30, 80 times higher performance-per-watt than contemporary CPUs and GPUs. TPUs are designed for high volume low precision computation as little as 8-bit precision without hardware for rasterization or texture mapping. They achieve more input output operations per joule compared to graphics processing units.

  124041

• 3. Generational Evolution And Specifications

  ›

  The second-generation TPU was announced in May 2017 using 16 GB of High Bandwidth Memory increasing bandwidth to 600 GB/s and performance to 45 teraFLOPS. The third-generation TPU arrived on the 8th of May 2018 with processors twice as powerful as the second generation deployed in pods with four times as many chips. An April 2023 paper claimed TPU v4 is 5, 87% faster than a Nvidia A100 at machine learning benchmarks. Google announced Trillium in May 2024 claiming a 4.7 times performance increase relative to TPU v5e via larger matrix multiplication units. In April 2025, Google unveiled TPU v7 called Ironwood which will have a peak computational performance rate of 4,614 TFLOP/s. Each generation increased memory capacity from 8 GiB DDR3 to 192 GB HBM while boosting clock speeds from 700 MHz to 1750 MHz.

  465557

• 4. Cloud Infrastructure And Commercial Access

  ›

  On the 12th of February 2018, The New York Times reported that Google would allow other companies to buy access to those chips through its cloud-computing service. Google provides third parties access to TPUs through its Cloud TPU service as part of the Google Cloud Platform and through notebook-based services Kaggle and Colaboratory. Broadcom is a co-developer translating Google's architecture into manufacturable silicon covering all generations since the program's inception. In September 2025, Google was in talks with several neoclouds including Crusoe and CoreWeave about deploying TPU in their datacenter. By November 2025, Meta was in talks with Google to deploy TPUs in its AI datacenters. Some models are commercially available while others remain proprietary to Google's internal infrastructure.

  165953

• 5. Edge Computing And Mobile Integration

  ›

  In July 2018, Google announced the Edge TPU designed to run machine learning models for edge computing meaning it consumes far less power compared to datacenter hosted units. The Edge TPU is capable of 4 trillion operations per second with 2 W of electrical power. On the 15th of October 2019, Google announced the Pixel 4 smartphone which contains an Edge TPU called the Pixel Neural Core customized to meet key camera feature requirements. Google followed this by integrating an Edge TPU into a custom system-on-chip named Google Tensor released in 2021 with the Pixel 6 line of smartphones. Product offerings include single-board computers like the Coral Dev Board running Mendel Linux OS and USB accessories supporting Debian-based systems on x86-64 and ARM64 hosts.

  607172

• 6. Legal Challenges And Patent Disputes

  ›

  In 2019, Singular Computing filed suit against Google alleging patent infringement in TPU chips regarding floating-point number dynamic ranges. By 2020, Google had successfully lowered the number of claims the court would consider to just two claiming a dynamic range of 10^-6 to 10^6 for floating point numbers. In a 2023 court filing, Singular Computing specifically called out Google's use of bfloat16 as that exceeds the dynamic range of float16. By January 2024, subsequent lawsuits brought the number of patents being litigated up to eight. Towards the end of the trial later that month, Google agreed to a settlement with undisclosed terms. The standard float16 format only has five bits for the exponent which cannot achieve the claimed dynamic range without resorting to subnormal numbers.

  757879