A large language model (LLM) is a computer program that has learned to predict the next fragment of text from the fragments before it, and has been trained to do so on a body of writing so large that the prediction becomes useful for drafting, summarising, translating, coding and answering questions. It exists because a single statistical model trained on general text turned out to perform many language tasks that once each needed a purpose-built system. ChatGPT, Google's Gemini, Anthropic's Claude and Meta's Llama are all LLMs, and so are the components that now sit inside search engines, ad platforms and social feeds.

"Large" refers to two quantities: the number of parameters, the adjustable numbers inside the model that store what it has learned, and the volume of training text. Neither has an agreed threshold. A model with a few billion parameters counts as "small" today.

How an LLM produces text

The model never sees words. Text is first split into tokens, chunks of characters drawn from a fixed vocabulary that commonly holds 50,000 to 200,000 entries. In English, one token averages about four characters, so 100 tokens correspond to roughly 75 words, according to OpenAI's help centre. Each token is converted into a long list of numbers, an embedding, that places it in a mathematical space where related meanings sit close together.

Those numbers then pass through a stack of identical layers built on the transformer architecture. The key operation is attention: at each layer, every token computes how much weight to give every earlier token when updating its own representation. That is how the model links "it" to a noun three sentences back.

The final layer outputs a probability for every token in the vocabulary. A sampling step picks one, guided by settings such as temperature, where a value near zero favours the most likely token and higher values allow less likely choices. The chosen token is appended to the input and the whole process repeats, one token at a time, until the model emits a stop token or hits a length limit. That runtime process is called inference, and it is where most of the operating cost of an LLM sits.

Everything the model can consider at once must fit in its context window, measured in tokens. The original GPT-3 in 2020 accepted 2,048 tokens. Google announced a one million token window for Gemini 1.5 Pro on February 15, 2024. The input a person or program supplies is the prompt, and a few worked examples placed inside it can teach a task without retraining, a technique known as few-shot prompting.

How a model is trained

Training happens in stages.

  • Pre-training. The model reads trillions of tokens of web pages, books, code and other text, and its parameters are adjusted billions of times to make its next-token guesses less wrong. This stage consumes most of the compute and produces a "base model" that continues text but does not reliably follow instructions.
  • Fine-tuning. Developers then train on curated examples of questions and good answers, so the model behaves as an assistant.
  • Preference training. Human raters rank alternative answers, and the model is tuned towards the preferred ones, a method called reinforcement learning from human feedback (RLHF). OpenAI's InstructGPT paper of March 2022 reported that raters preferred outputs from a 1.3 billion parameter tuned model over those of the 175 billion parameter GPT-3.

Since September 2024, developers have added a further stage for reasoning models, which are trained to generate long internal chains of intermediate steps before answering. Those hidden tokens are billed at output rates.

The output of all this is a static file of parameters. An LLM does not learn from each conversation, and its knowledge stops at a training cut-off date. Products work around that limit with retrieval-augmented generation, fetching documents at the moment of a question and writing from them, a step usually called grounding.

Origin and evolution

Language models are older than computers that could run them well. Claude Shannon used statistical approximations of English built from letter and word sequences in his 1948 paper on information theory. For decades, n-gram models, which count how often short word sequences appear, powered speech recognition and machine translation. The phrase itself appears in the title of a June 2007 Google paper, "Large Language Models in Machine Translation", by Thorsten Brants, Jeffrey Dean and colleagues, which described n-gram models trained on up to two trillion tokens.

The modern meaning dates from June 2017, when eight Google researchers published "Attention Is All You Need", introducing the transformer. OpenAI's GPT-1 followed in June 2018 with 117 million parameters, and Google released BERT in October 2018, deploying it in Search a year later. OpenAI announced GPT-2, with 1.5 billion parameters, on February 14, 2019, initially withholding the full model over misuse concerns. GPT-3, described on May 28, 2020, had 175 billion parameters and showed that one model could perform tasks from a handful of examples.

Two research results then shaped how models were built. OpenAI's scaling laws paper of January 2020 found that performance improved predictably with more parameters, data and compute. DeepMind's Chinchilla paper of March 2022 showed that many models had been undertrained, and that a 70 billion parameter model trained on 1.4 trillion tokens beat much larger rivals, implying roughly 20 tokens per parameter.

ChatGPT launched on November 30, 2022, putting a fine-tuned LLM in front of the public. GPT-4 arrived on March 14, 2023 without a disclosed parameter count. Meta's Llama, released in February 2023, and Llama 2, licensed for commercial use in July 2023, opened a parallel track of downloadable "open-weight" models. The most capable systems at any moment are now described as frontier models, a label that carries legal duties in the EU and some US states.

Why it matters for marketing

LLMs have moved from a product category into the infrastructure of advertising and search. Testimony from Google engineer Pandu Nayak, made public in May 2025, said Google was rethinking its search stack "from the ground-up" with LLMs taking a more prominent role. LinkedIn rebuilt its feed retrieval and ranking around LLM-based models in March 2026, for a platform of 1.3 billion members.

The models are also becoming ad inventory. ChatGPT ads began as a pilot on February 9, 2026, and the Ads Manager opened to all US businesses on May 5, 2026.

Inside agencies and platforms, LLMs handle planning, setup, reporting and analysis. A Digiday report covered in January 2026 found the industry unwilling to hand LLMs "the keys" to bidding and budgets, with executives citing the mismatch between probabilistic outputs and deterministic auction logic.

Limitations and disputes

Fabrication is the most persistent weakness. Because an LLM produces plausible continuations rather than looking up facts, it can state falsehoods fluently, a failure known as hallucination. Gracenote found that an ungrounded model fully fabricated metadata for 19.5% of 2,600 streaming titles. Researchers disagree on whether the problem can be engineered away or is inherent to next-token prediction.

Training data is contested in court. In June 2025, Judge William Alsup ruled that training Claude on books was fair use but that downloading pirated copies was not, and Anthropic then agreed to a $1.5 billion settlement covering roughly 500,000 works. Two days after the Alsup ruling, Judge Vince Chhabria found for Meta against 13 authors, while warning that better evidence of market harm could change outcomes. Privacy regulators raise parallel questions; the European Data Protection Board's ChatGPT taskforce flagged web scraping, accuracy and data subject rights in May 2024. Once data is absorbed, it is hard to remove: Tom Chavez of super{set} argued in August 2026 that enterprise data cannot be taken out of a trained LLM, because machine unlearning remains unsolved.

Understanding is disputed. A 2021 paper by Emily Bender, Timnit Gebru and co-authors described LLMs as "stochastic parrots" that stitch together linguistic forms without reference to meaning. Others cite benchmark results in law and code as evidence of real capability. Security adds a newer risk: an LLM cannot reliably separate instructions from data, so text hidden in a web page can redirect it, an attack called prompt injection.

Scale itself is questioned. NVIDIA researchers argued in June 2025 that models under 10 billion parameters could handle 60% to 80% of agentic tasks now sent to far larger systems.

Not the same as

  • Generative AI is the broader category. Image, video and music generators are generative AI but not language models.
  • Chatbot names a product. ChatGPT is a chatbot application built on top of LLMs; the model is the engine, and the chatbot adds an interface, memory, tools and safety filters.
  • Foundation model is a term coined at Stanford in August 2021 for any large model trained on broad data and adapted to many tasks. Most LLMs are foundation models, but foundation models also include vision and multimodal systems.
  • AI agent describes software that uses an LLM to plan and take actions, such as calling tools or buying media. The LLM is one component of the agent.

Recent developments

Price competition has dominated 2026. Google released Gemini 3.6 Flash in July at $1.50 per million input tokens, using 17% fewer output tokens than its predecessor. Anthropic released Claude Haiku 5.5 on October 7, 2026, priced 90% below Haiku 4.5 for prompts up to 100,000 tokens, a list-price comparison rather than a measured workload cost. As of October 2026, cheaper and smaller models are spreading alongside a handful of very large ones.

Timeline

  • 1948 - Claude Shannon publishes "A Mathematical Theory of Communication", using statistical approximations of English text.
  • June 2007 - Google researchers publish "Large Language Models in Machine Translation", training n-gram models on up to two trillion tokens.
  • June 12, 2017 - "Attention Is All You Need" introduces the transformer architecture.
  • June 11, 2018 - OpenAI releases GPT-1 with 117 million parameters.
  • October 11, 2018 - Google publishes BERT.
  • February 14, 2019 - OpenAI announces GPT-2 with 1.5 billion parameters, initially withholding the full model.
  • January 23, 2020 - OpenAI publishes scaling laws for neural language models.
  • May 28, 2020 - OpenAI describes GPT-3, with 175 billion parameters.
  • March 2021 - Bender, Gebru and co-authors publish "On the Dangers of Stochastic Parrots".
  • August 2021 - Stanford researchers coin "foundation model".
  • March 2022 - OpenAI's InstructGPT paper describes RLHF; DeepMind's Chinchilla paper revises scaling rules.
  • November 30, 2022 - ChatGPT launches.
  • February 24, 2023 - Meta releases Llama.
  • March 14, 2023 - OpenAI releases GPT-4.
  • July 18, 2023 - Meta releases Llama 2 for commercial use.
  • February 15, 2024 - Google announces a one million token context window for Gemini 1.5 Pro.
  • May 2024 - The EDPB ChatGPT taskforce publishes its report.
  • September 12, 2024 - OpenAI releases o1, its first reasoning model.
  • June 23 and 25, 2025 - US federal judges rule on fair use in Bartz v. Anthropic and Kadrey v. Meta.
  • August 7, 2025 - OpenAI launches GPT-5.
  • September 2025 - Anthropic agrees a $1.5 billion copyright settlement.
  • February 9, 2026 - ChatGPT ads pilot begins.
  • May 5, 2026 - ChatGPT Ads Manager opens to all US businesses.
  • October 7, 2026 - Anthropic releases Claude Haiku 5.5.

Summary

Who: LLMs are built by laboratories including OpenAI, Google, Anthropic, Meta and DeepSeek, run on hyperscale cloud infrastructure, and are used by search engines, ad platforms, agencies and advertisers.

What: A large language model is a transformer-based neural network trained on trillions of tokens to predict the next token, then fine-tuned to follow instructions, producing text one token at a time.

When: The phrase appeared in a 2007 Google paper on n-gram models; its modern meaning follows the transformer of June 2017, GPT-3 in May 2020 and ChatGPT's launch on November 30, 2022.

Where: LLMs run in data centres and reach users through chat products, APIs, search features such as AI Overviews, social feed ranking and, since February 2026, ChatGPT's ad inventory.

Why: One general model trained on broad text can perform many language tasks that once required separate systems, which explains its spread into search and advertising, and also its disputes over accuracy, copyright and cost.