Short answers, pulled from the story.
A large language model is a neural network trained on vast amounts of text to perform natural language tasks, especially generating, summarizing, translating, and analyzing text. As of 2026, the most capable LLMs use transformer architectures, which process relationships between all tokens in a sequence simultaneously using an attention mechanism, making them more efficient than earlier recurrent models.
Google researchers introduced the transformer architecture at the NeurIPS conference in 2017, in a landmark paper titled "Attention Is All You Need". The architecture was based on the attention mechanism developed by Bahdanau et al. in 2014 and was designed to improve upon sequence-to-sequence technology from 2014.
Training costs vary widely by model size. Training GPT-2, a 1.5-billion-parameter model, cost $50,000 in 2019. Training PaLM, a 540-billion-parameter model, cost $8 million in 2022. Megatron-Turing NLG 530B cost around $11 million in 2021.
Hallucination refers to LLMs generating text that is syntactically fluent and sounds factually plausible but is internally inconsistent with training data or factually incorrect. A 2023 study found ChatGPT 3.5 Turbo would begin outputting verbatim excerpts from training data after a few hundred repetitions of the same word.
DeepSeek R1 is a 671-billion-parameter open-weight reasoning model released by Chinese company DeepSeek in January 2025. It achieved performance comparable to OpenAI's o1 while being significantly more cost-effective to operate per token, and its weights were made publicly available for researchers to study, though its training data remained private.
According to a 2024 study by Luccioni, Jernite, and Strubell, simple classification tasks average 0.002 to 0.007 watt-hours per prompt. Text generation and summarization each average around 0.05 watt-hours per prompt. Image generation is most energy-intensive, averaging 2.91 watt-hours per prompt, with the least efficient model using 11.49 watt-hours per image.