Hallucination (artificial intelligence)

• 1. Introduction

  ›

  AI hallucination is the tendency of artificial intelligence systems to generate false or misleading information presented as fact. Professor Ethan Mollick of Wharton put it memorably. He called ChatGPT an "omniscient, eager-to-please intern who sometimes lies to you". Data scientist Teresa Kubacka devised a more clinical test. She invented the phrase "cycloidal inverted electromagnon", a term she knew to be entirely fictional, and asked ChatGPT to explain it. The system returned a confident, detailed answer backed by plausible-looking citations. Kubacka found herself double-checking whether she had accidentally coined a real term. Oren Etzioni, after similar experiments, concluded that such systems can often give "a very impressive-sounding answer that's just dead wrong". These failures are not random noise. They reflect something structural in how the models work, and they carry consequences that extend well beyond inconvenient chatbot exchanges. Where did the word "hallucination" come from, and why has its use provoked genuine scientific controversy? What causes a machine to fabricate with such confidence? And what happens when hallucinated text finds its way into courtrooms, government reports, and research journals?

  49339

• 2. A Word's Origins

  ›

  Eric Mjolsness gave the term its first documented AI use in his 1986 PhD thesis, though in that context it described something beneficial. In computer vision, hallucination referred to the process of adding detail to an image. The idea was to generate a version richer or higher-resolution than the raw data would support. Simon Baker and Takeo Kanade built on this with their face hallucination algorithm, published in 1999. Their method produced high-resolution face images from blurry low-resolution inputs.

  Through the 2000s, the term migrated into statistical machine translation, where it picked up a negative meaning. A translation model would sometimes produce output bearing no meaningful connection to the source text. Researchers borrowed the imaging term to name this failure mode. In 2015, Saurabh Gupta and Jitendra Malik identified hallucinations in visual semantic role labeling tasks. That same year, computer scientist Andrej Karpathy used the word "hallucinated" in a blog post. He was describing his recurrent neural network language model generating an incorrect citation link. By 2017, Google researchers had applied the term to neural machine translation models generating responses unrelated to their source text. In 2018, it entered computer vision again, describing objects erroneously detected because of adversarial attacks.

  Meta brought the word to a far wider audience in July 2021. Releasing BlenderBot 2, the company warned that the system was prone to "hallucinations", which it defined as "confident statements that are not true". On the 15th of November 2022, Meta released Galactica, designed to "store, combine and reason about scientific knowledge". When asked to draft a paper about creating avatars, Galactica cited a fictitious paper attributed to a real author who worked in the field. Meta withdrew Galactica on the 17th of November due to offensiveness and inaccuracy. OpenAI's ChatGPT launched on the 30th of November 2022. The New York Times and many other news outlets adopted "hallucinations" as their shorthand for the model's frequent errors.

  By 2023, the Cambridge dictionary had updated its definition to include this AI-specific sense. Many researchers thought that recognition had arrived far too early for a term they found deeply misleading.

  1012141517

• 3. A Contested Vocabulary

  ›

  Mary Shaw's response to the word was blunt. She wrote that the trend was "appalling", arguing that it anthropomorphizes software and frames objectively incorrect outputs as mere idiosyncratic quirks of the system. Usama Fayyad, executive director of the Institute for Experimental Artificial Intelligence at Northeastern University, made a related case. He argued the term misleadingly personified large language models and was too vague to be scientifically useful.

  In Salon, statistician Gary Smith argued that large language models "do not understand what words mean", making the hallucination metaphor an unreasonable anthropomorphism. Murray Shanahan has argued that terms like "hallucination" encourage users to attribute cognitive processes to systems that do not have them. These systems operate through statistical pattern completion, Shanahan notes, not through reasoning or understanding. Kristina Sekrst raised a parallel concern, arguing that psychological vocabulary obscures the difference between the appearance of mental properties and their genuine presence. Forster and Skop went further, asserting that tech companies deploy the hallucination metaphor specifically to deflect responsibility for non-factual outputs.

  Journalist Benj Edwards, writing in Ars Technica, took a more constructive approach. He acknowledged the word was controversial but argued that some form of metaphor remained necessary. Edwards suggested "confabulation" as a better alternative, to capture processes that involve "creative gap-filling". Hicks, Humphries, and Slater made a sharper claim in the journal Ethics and Information Technology. They argued that LLM output qualifies as "bullshit" under philosopher Harry Frankfurt's definition of the term. They found these models to be "in an important way indifferent to the truth of their outputs", with true statements only accidentally true and false ones only accidentally false. Some researchers have also adopted the term "botshit", often referring to uncritical use of AI-produced content.

  OpenAI offered its own definitions in May 2023. In one document it described hallucination as "a tendency to invent facts in moments of uncertainty". In another it called them "a model's logical mistakes". A July 2024 White House report on fostering public trust in AI research mentioned hallucinations only in the context of reducing them. When a Nobel committee recognized prize-winning research using AI to design proteins, they chose not to use the word at all. They called the outputs "imaginative protein creation" instead.

  Symbolic artificial intelligence models generally do not produce hallucinations at all. Some researchers cite this distinction to argue the problem is architectural rather than inherent to AI as a whole.

  127283132

• 4. Inside the Machine

  ›

  In 2025, interpretability researchers at Anthropic identified an internal mechanism inside Claude that sheds light on why hallucinations occur. Their work found circuits that ordinarily cause the model to decline answering unless it has enough reliable information to draw on. When those circuits function normally, they suppress responses the model cannot support. When they malfunction, for instance when the model recognizes a name but lacks genuine knowledge about that person, it generates plausible-sounding responses anyway.

  Generative pretrained transformer models learn through a process of predicting the next word in a sequence. That training consistently rewards making a plausible guess rather than staying silent. When a model lacks reliable information, the training still pushes it toward producing a response. Some researchers frame this using creativity theory. Amabile and Pratt defined human creativity as the production of novel and useful ideas. Applied to machines, a focus on novelty can produce original but inaccurate responses. A focus on usefulness tends to produce memorized content lacking originality. Hallucination sits in the space between those two pressures.

  Research published in 2022 found that language models not only hallucinate but also amplify existing hallucinations. This was true even in models specifically designed to reduce the problem. Incomplete, inaccurate, or unrepresentative training data is another source of error. When a model learns from flawed inputs, it tends to reproduce those flaws in predictable ways. OpenAI researchers pointed to how models are scored on benchmarks as part of the cause. Systems that reward guessing over acknowledging uncertainty contribute directly to the behavior, they found.

  Nvidia Guardrails, launched in 2023, gives developers a way to hard-code certain responses via script rather than leaving them to the model. Researchers have also proposed having multiple chatbots debate one another until they reach a consensus answer. Another approach pairs language models with formal knowledge bases and inference engines. One barrier to broader adoption is behavioral: in consumer applications, users tend to prefer rapid, overconfident answers over cautious, uncertainty-aware ones.

  By 2023, a Google executive had identified hallucination reduction as a "fundamental" task for Google Gemini, the company's competitor to ChatGPT.

  4036383723

• 5. When AI Goes to Court

  ›

  On the 23rd of June 2023, judge P. Kevin Castel dismissed the Mata v. Avianca case and issued a $5,000 fine for bad faith conduct. Lawyer Stephen Schwartz had submitted six fictitious case precedents generated by ChatGPT in his personal injury brief. Schwartz said he had never previously used ChatGPT and had not understood it could fabricate legal citations. When the nonexistence of the citations was exposed, ChatGPT continued to assert their authenticity. Castel characterized one cited opinion as "gibberish" and "bordering on nonsensical". Brantley Starr of the Northern District of Texas issued a standing rule in response. He banned unreviewed filings generated by AI, writing that such platforms "make stuff up, even quotes and citations".

  Air Canada's support chatbot had told a customer that bereavement discounts could be claimed retroactively. The chatbot said such claims were valid within 90 days of the date the ticket was issued. No such policy existed. In February 2024, Canada's Civil Resolution Tribunal ordered the airline to pay damages and honor the invented terms. Air Canada's defense, that the chatbot was a "separate legal entity that is responsible for its own actions", was rejected by the tribunal.

  In October 2025, an A$440,000 Deloitte report to the Australian government was found to contain fabricated citations. The errors included non-existent academic sources and a false quote attributed to a federal court judgment. Deloitte submitted a revised report and agreed to issue a partial refund. Within weeks, The Independent, a news publication in Newfoundland and Labrador, found that a separate Deloitte report also contained hallucinated citations. That report was a CA$1.6 million Health Human Resources Plan for Newfoundland and Labrador, commissioned in May 2025. It contained at least four references to non-existent research papers.

  Damien Charlotin of HEC Paris and Sciences Po began tracking these incidents in April 2025. His database, called "AI Hallucination Cases", listed over 1,300 instances of hallucinations in legal decisions within its first year. On the 18th of March 2026, Assistant District Attorney Deborah Leslie appeared before the Supreme Court of Georgia. She was there to argue against an appeal in a murder conviction. Chief Justice Nels S. D. Peterson identified at least five fictitious citations in her filing. He also found five others that did not support the propositions for which they were cited. On the 30th of March, Leslie admitted in an affidavit to using research generated by AI without verifying it. District Attorney Tasha Mosley issued a written apology to Peterson. On the 5th of May, the Supreme Court vacated the lower court's ruling. Leslie was suspended from practicing before the court for six months and ordered to complete additional education in ethics and the proper use of artificial intelligence.

  In April 2026, Sullivan & Cromwell submitted an apology to the U.S. Bankruptcy Court for the Southern District of New York. Opposing counsel Boies Schiller Flexner had discovered hallucinated citations in Sullivan & Cromwell's submission on behalf of client Prince Global Holdings Limited.

  7374565971

• 6. Science at the Edge

  ›

  A 2023 study in the Cureus Journal of Medical Science examined 178 references generated by GPT-3. Sixty-nine returned an incorrect or nonexistent digital object identifier. An additional 28 had no known identifier and could not be found in a Google search. Jerome Goddard of Mississippi State University documented an equally troubling case. ChatGPT had provided Goddard with information about ticks and cited a source. When he examined the source, both the digital object identifier and the author names had been fabricated. He contacted the named authors, who confirmed they had no knowledge of the paper's existence. Goddard concluded that physicians and biomedical researchers should not ask ChatGPT for sources or citations. Any references the system provides, he warned, must be carefully vetted for accuracy.

  A study analyzing 115 references from ChatGPT-3.5 found that 47% were fabricated outright. Another 46% cited real references but extracted incorrect information from them. Only 7% were both genuine and accurately cited. When pressed on its mistakes, ChatGPT sometimes corrects itself. Other times it continues to insist the response is correct and adds further misleading information.

  Researchers at Northwestern University of Chicago generated 50 abstracts based on existing reports and ran them through plagiarism-detection software. The software gave each abstract an originality score of 100%. Software designed to detect text produced by AI identified the fabricated abstracts accurately only 66% of the time. Research scientists achieved a detection rate of 68%, only marginally better.

  Nina Schick identified a broader democratic risk. She warned that AI could become "an engine to power all information and knowledge". The consequence, she argued, would be "degrading trust in the medium of digital information itself". Among the stranger downstream effects has been the emergence of the phrase "vegetative electron microscopy", which has appeared in multiple research papers after becoming embedded in AI training data.

  David Baker's laboratory at the University of Washington used AI hallucinations to design millions of proteins that do not occur in nature. The work generated roughly 100 patents and led to the founding of more than 20 biotech companies. Baker received the 2024 Nobel Prize in Chemistry for contributions that grew directly from this approach. At the California Institute of Technology, researchers used hallucinations to design a catheter that reduces bacterial contamination. The device features sawtooth-like spikes on its inner walls that prevent bacteria from gaining traction, potentially reducing the number of urinary tract infections. Anima Anandkumar, a professor at Caltech, has emphasized that these models are "taught physics" and that every output requires validation through rigorous testing. Researchers at Memorial Sloan Kettering Cancer Center have applied similar techniques to enhance blurry medical images. The University of Texas at Austin has used them to improve robot navigation systems. In meteorology, scientists generate thousands of subtle forecast variations to identify unexpected factors that can influence extreme weather events.

  JAMA has updated its submission policies to discourage citations produced by AI systems. The Journal of Cranio-Maxillofacial Surgery has called for automated citation-checking tools alongside AI literacy training for researchers and students.

  99100263464