Progress in artificial intelligence

• 1. Introduction

  ›

  Progress in artificial intelligence is a subject that has a strange habit of disappearing just as it succeeds. A quote that circulates widely in the field captures this: "A lot of cutting-edge AI has filtered into general applications, often without being called AI because once something becomes useful enough and common enough it's not labeled AI anymore." The technology blends into the background the moment it works. In the late 1990s and early 2000s, AI was already woven into the infrastructure of industries across the economy, yet the field received almost no credit for those achievements at the time. Researchers Andreas Kaplan and Michael Haenlein mapped the whole arc of what AI is trying to become: from narrow systems that can only do one thing, to general intelligence that can reason across domains it was never trained for, all the way to a hypothetical superintelligence that would match humans in creativity, social understanding, and wisdom. Games, exams, and competitions have become the measuring sticks along the way, each milestone forcing a reckoning with how much distance still remains.

• 2. Games as Proving Grounds

  ›

  Chess was the first great arena. An AI defeated a grandmaster in a regulation tournament game in 1988, and the same system, rebranded as Deep Blue, beat the reigning human world chess champion in 1997. By the 2010s, chess engines running on ordinary consumer hardware had left top human players far behind. The neural-network system AlphaZero then proved that superhuman play did not require human expert data at all; reinforcement learning from self-play alone was enough.

  Go was long considered the frontier that computers could not cross. Most observers expected superhuman performance in Go to be at least a decade away when AlphaGo defeated a European Go champion in October 2015. In March 2016, AlphaGo beat Lee Sedol, ranked among the world's top players. The prediction history for Go is a lesson in humility: in 1997, physicist Piet Hut estimated that machines would need until 2100 or later; Feng-Hsiung Hsu of Microsoft Research Asia forecast 2017 or sooner in 2007, and he was essentially right.

  Poker presented a different kind of challenge. Unlike chess or Go, poker is a game of imperfect information. Libratus resolved that benchmark with a victory in heads-up no-limit hold'em poker in 2017. Six-player no-limit hold'em followed in 2019. E-sports broadened the scope further: Facebook AI, DeepMind, and others engaged with the StarCraft franchise, and StarCraft II reached a high-human performance level in 2019. Gran Turismo Sport joined that list in 2022, showing that even real-time racing simulation had fallen within reach.

• 3. Exam Performance and Its Limits

  ›

  According to OpenAI, GPT-4 in 2023 scored around the 90th percentile on the Uniform Bar Exam, the 89th percentile on the mathematics section of the SAT, the 93rd percentile on SAT Reading and Writing, and the 99th percentile on GRE verbal reasoning. On the 2020 USA Biology Olympiad semifinal exam, it scored in the 99th to 100th percentile. These were numbers that, just a few years earlier, would have seemed implausible for any machine.

  Independent researchers found that ChatGPT, based on GPT-3.5, performed at or near the passing threshold on all three parts of the United States Medical Licensing Examination in 2023, without any domain-specific fine-tuning. GPT-3.5 also earned a low but passing grade on examinations for four law school courses at the University of Minnesota.

  By 2025, a more complicated picture had emerged. A benchmarking study on publicly available USMLE sample questions found that models such as ChatGPT and DeepSeek outperformed some rivals but still made distinct errors and showed persistent limitations in clinical reasoning. On the legal side, the LEXam benchmark, built from 340 law exams across 116 law school courses, found that long-form legal reasoning remained genuinely difficult for large language models, especially on open-ended questions requiring structured, multi-step analysis.

  Stanford HAI cautioned in 2025 that benchmark and exam performance should not be treated as equivalent to reliable real-world performance or trustworthy decision-making. A 2026 Nature paper introducing a benchmark called Humanity's Last Exam noted that state-of-the-art systems had surpassed 90% accuracy on several popular benchmarks, while still scoring low on a harder test designed to probe the frontier of expert human knowledge.

• 4. Where Machines Still Fall Short

  ›

  Researcher Andrew Ng offered a rule of thumb that captures something real: "almost anything a typical human can do with less than one second of mental thought, we can probably now or in the near future automate using AI." The corollary, left unspoken, is that everything requiring more than a second of reflection is still contested ground.

  Some version of Moravec's paradox holds that humans are more likely to outperform machines in areas shaped directly by natural selection, including physical dexterity. As of 2017, bipedal robots could walk but remained less stable than human walkers. Speech recognition reached a level described as "nearly equal to human performance" in 2017, though it still sat in the sub-human category at that point. Medical systems in that era could diagnose certain conditions well but could not explain to users why they had reached a given conclusion, a gap that matters enormously in clinical settings.

  The ConceptARC benchmark put a number on the gap that persisted into 2023. Both GPT-4 and other models tested that year scored around 60% on most categories, while humans scored 91% on all categories. Later research in 2025 complicated that comparison: human-generated output grids turned out to be accurate only 73% of the time, while AI models available that year managed to score above 77%, suggesting the benchmark's assumptions about a fixed human standard needed revision.

  Translation, word-sense disambiguation, object recognition, and the Angry Birds video game all remained in the sub-human tier as of 2020. The broader category of tasks difficult to solve without contextual knowledge continued to present genuine obstacles.

• 5. Forecasting AGI

  ›

  Herbert Simon, an AI pioneer and economist, predicted in 1965 that machines would be capable within twenty years of doing any work a human could do. Marvin Minsky wrote in 1970 that within a generation the problem of creating artificial intelligence would be substantially solved. Both predictions missed by decades.

  Four polls conducted in 2012 and 2013 placed the median expert estimate for the arrival of human-level artificial general intelligence somewhere between 2040 and 2050. A poll conducted around 2016 by Katja Grace of the Future of Humanity Institute found that the answer depended heavily on how the question was framed. When respondents were asked when unaided machines could accomplish every task better and more cheaply than human workers, the aggregated median answer was 45 years. When the question shifted to full automation of all occupations, the median jumped to 122 years. The median estimate for full automation of the role of AI researcher itself was around 90 years. Asian researchers were considerably more optimistic than North American researchers: Asians predicted around 30 years on average for machines to accomplish every task, compared with 74 years predicted by North Americans.

  A larger survey of 2,778 researchers who had published in top AI venues, fielded in 2023 and published in 2025, put a 10% probability on unaided machines outperforming humans at every task by 2027 and a 50% probability by 2047. Full automation of all human occupations was estimated to reach a 10% probability by 2037. Despite these compressed timelines, Stanford HAI co-director James Landay stated plainly in predictions for 2026: "there will be no AGI this year."