Supervised fine-tuning (SFT) is the stage in which a pretrained language model is trained further on a set of example prompts paired with the answers it should give. The model reads each prompt, is shown the target answer, and has its internal weights adjusted so that answer becomes more likely next time. SFT exists because pretraining on web text produces a model that continues documents rather than one that answers questions. A few thousand curated demonstrations are usually enough to turn it into something that behaves like an assistant.

It is "supervised" in the classic machine learning sense: every example carries a correct answer rather than a score or a preference. That separates it from the reinforcement learning stages that usually follow, and from prompting, which changes nothing inside the model.

How a fine-tuning run works

The raw material is a dataset of demonstrations. Each record holds an input and the desired output, typically formatted as a conversation with system, user and assistant turns. OpenAI's API documentation specifies JSON Lines (JSONL), one chat object per line, with a messages array and optional tools definitions so that a model can also be taught when and how to call a function. The same documentation sets a minimum of 10 examples and suggests starting with about 50 well-crafted ones, with gains typically visible at 50 to 100. General assistants are trained on tens of thousands to millions of pairs, written by contractors or generated by other models.

Training then proceeds as it did during pretraining, with one change. The model still predicts the next token and is penalised by a cross-entropy loss when it guesses wrong. In SFT, however, the loss is normally counted only on the answer tokens. The prompt is visible to the model but it is not graded on reproducing it. The practical controls are the number of epochs (full passes over the data, commonly one to four), the learning rate, which is set well below pretraining values to avoid erasing what the model already knows, and the batch size. A held-out validation set detects overfitting, where the model memorises examples instead of learning the pattern.

There are two ways to update the weights. Full fine-tuning adjusts every parameter, which for a model with tens of billions of parameters requires the same class of hardware used to train it. Parameter-efficient fine-tuning (PEFT) freezes the original weights and trains a small add-on. The dominant method is LoRA (low-rank adaptation), published by Edward Hu and colleagues at Microsoft in June 2021. LoRA inserts pairs of small matrices alongside existing layers and trains only those. On GPT-3 175B, the paper reported that LoRA cut trainable parameters by 10,000 times and GPU memory requirements by three times compared with full fine-tuning. The adapter is megabytes rather than gigabytes, so one base model can serve many customers, each with its own adapter. QLoRA, published by Tim Dettmers and colleagues at the University of Washington in May 2023, combined LoRA with 4-bit compression of the frozen model, making it possible to fine-tune a 65-billion-parameter model on a single 48GB graphics card.

Hosted services hide most of this. A customer uploads a file, selects a base model, and receives a private model identifier for inference. OpenAI's service assesses each completed model across 13 safety categories and can block deployment if too many outputs fail in one, according to its documentation.

Where SFT sits in the training pipeline

A modern large language model is built in stages. Pretraining consumes trillions of tokens of text and produces a base model. Post-training then shapes behaviour, and SFT is almost always its first step. After it come preference or reward stages: reinforcement learning from human feedback (RLHF), its simpler successor Direct Preference Optimization (DPO), or reinforcement learning against verifiable rewards for maths and code.

The order matters. Meta's content moderation team reported in December 2025 that applying reinforcement learning directly to base models, without SFT first, led to severe degradation in instruction adherence. Starting from SFT stabilised training, though the two-stage approach introduced more factual hallucination. When OpenAI released GPT-5 in August 2025, it described post-training that includes supervised fine-tuning stages and high-compute reinforcement learning.

Origin and evolution

Fine-tuning a pretrained network is old practice in computer vision. For language, the decisive year was 2018. Jeremy Howard and Sebastian Ruder's ULMFiT paper, in January, showed that a language model pretrained on general text could be fine-tuned for classification with little labelled data. OpenAI's first GPT paper, in June, paired "generative pre-training" with supervised fine-tuning on each target task. Google's BERT followed in October. Fine-tuning then meant one model per task.

Instruction tuning changed the target from a single task to instruction-following in general. Google's FLAN paper, first posted on September 3, 2021, fine-tuned a 137-billion-parameter model on more than 60 natural language tasks phrased as instructions. According to the paper, the result beat zero-shot GPT-3 175B on 20 of 25 tasks evaluated.

The pipeline used today was set out in OpenAI's InstructGPT paper, "Training language models to follow instructions with human feedback", posted on March 4, 2022 by Long Ouyang and colleagues. Labellers wrote demonstrations of desired answers, which were used for SFT; they then ranked model outputs to train a reward model, against which the SFT model was optimised with reinforcement learning. Outputs from the 1.3-billion-parameter InstructGPT were preferred to those of the 175-billion-parameter GPT-3, "despite having 100x fewer parameters", according to the abstract. ChatGPT, released on November 30, 2022, applied the same recipe.

The open-source wave of 2023 pushed in the other direction, towards less data. Meta researchers' LIMA paper, posted on May 18, 2023, fine-tuned a 65-billion-parameter LLaMA model with the standard supervised loss on only 1,000 curated prompts and responses. Its answers were judged equivalent or preferred to GPT-4's in 43% of cases, according to the paper.

Why it matters to marketers

For advertisers and agencies, SFT is the mechanism behind the claim that an AI system "knows" a brand. Fine-tuning a model on approved copy, customer service transcripts or classification labels bakes the pattern into the weights, so it no longer has to be restated in every prompt. That cuts tokens per request and lets a smaller model do a narrow job. NVIDIA researchers made that case in 2025, arguing that small fine-tuned models can replace large ones for many agent tasks.

Ad platforms use the technique internally. Meta's AdLlama work, posted in July 2025, compared a generative ad text model trained with reinforcement learning on ad performance data against "a supervised imitation model trained on curated ads". In a 10-week test across nearly 35,000 advertisers and 640,000 ad variations, the reinforcement-trained model lifted click-through rates by 6.7%, according to the paper. The SFT model was the baseline, not the winner.

Limitations and disputes

SFT teaches a model to imitate, and imitation has limits. A January 2025 study by researchers including Sergey Levine and Quoc Le, titled "SFT Memorizes, RL Generalizes", found that supervised fine-tuning "tends to memorize training data and struggles to generalize" to out-of-distribution cases, while reinforcement learning generalised better. Meta's moderation team reported 10 to 100 times better data efficiency from reinforcement learning than from SFT on its classification tasks.

The superficial alignment argument, made by the LIMA authors, cuts the other way: if 1,000 examples suffice, SFT mostly teaches format and style, not knowledge. That raises a practical risk. Fine-tuning on facts a model never learned in pretraining can encourage it to state things confidently that it does not know, a route to hallucination. Planning research found that fine-tuning showed limited improvement in reasoning ability.

Catastrophic forgetting, the erosion of general abilities after narrow training, is a documented cost. Fine-tuning can also strip safety behaviour from open-weight models. And a fine-tune is pinned to its base: when the base model is retired, the fine-tune goes with it.

SFT and adjacent terms

  • In-context learning and few-shot prompting place examples in the prompt. Weights stay frozen and the effect ends with the call. SFT changes weights permanently.
  • RLHF and DPO train on preferences between answers, or on rewards, rather than on a single correct answer. They usually start from an SFT model.
  • Instruction tuning is SFT on instruction-formatted data aimed at general instruction-following. Most SFT of chatbots today is instruction tuning; SFT on a single classification task is not.
  • Knowledge distillation becomes SFT when a large teacher model writes the training answers for a smaller student. Imitation learning, the broader robotics and control term, covers any learning from demonstrations; SFT is its language-model form.

Recent developments

The commercial market for hosted fine-tuning has split. Thinking Machines Lab, founded by former OpenAI chief technology officer Mira Murati, launched Tinker, an API for fine-tuning open-weight models with LoRA, in October 2025, and made it generally available on December 12, 2025. OpenAI moved the other way. On May 7, 2026 it told developers it was winding down its self-serve fine-tuning platform, closing it to organisations that had never fine-tuned. From July 2, 2026, job creation required inference on a fine-tuned model in the previous 60 days. Fine-tuned versions of GPT-4.1 nano were shut down on October 23, 2026. As of November 2026, existing customers can create jobs until January 6, 2027, and fine-tunes of GPT-4o, GPT-4o mini, GPT-4.1 and GPT-4.1 mini stop with their base models on January 14, 2027, according to OpenAI's deprecations page.

Inside the labs, SFT remains standard. Research published in January 2026 on the "assistant axis" described how post-training through supervised fine-tuning, RLHF and constitutional training instils the helpful assistant identity users meet.

Timeline

  • January 2018 - Howard and Ruder publish ULMFiT, showing that pretrained language models can be fine-tuned for classification.
  • June 2018 - OpenAI's first GPT paper pairs generative pretraining with supervised fine-tuning per task.
  • October 2018 - Google releases BERT, popularising the pretrain-then-fine-tune pattern.
  • June 17, 2021 - Microsoft researchers post the LoRA paper.
  • September 3, 2021 - Google posts FLAN, establishing instruction tuning across more than 60 tasks.
  • March 4, 2022 - OpenAI posts the InstructGPT paper, setting out SFT followed by RLHF.
  • November 30, 2022 - ChatGPT launches, built on the InstructGPT recipe.
  • May 18, 2023 - LIMA shows strong results from SFT on 1,000 curated examples.
  • May 23, 2023 - QLoRA enables 65B fine-tuning on one 48GB GPU.
  • May 29, 2023 - DPO is published as a lighter alternative to RLHF after SFT.
  • January 28, 2025 - "SFT Memorizes, RL Generalizes" is posted.
  • July 29, 2025 - Meta researchers post AdLlama results against an SFT baseline.
  • October 2025 - Thinking Machines Lab launches Tinker; general availability follows December 12, 2025.
  • May 7, 2026 - OpenAI announces it is winding down self-serve fine-tuning.
  • October 23, 2026 - OpenAI shuts down GPT-4.1 nano fine-tunes.
  • January 6, 2027 - Scheduled end of new OpenAI fine-tuning jobs.
  • January 14, 2027 - Scheduled shutdown of GPT-4o and GPT-4.1 family fine-tunes.

Summary

Who: Model developers such as OpenAI, Google, Meta and Anthropic use SFT in post-training. Enterprises, agencies and ad platforms fine-tune models through hosted services or open-weight tooling, with contractors and other models supplying the training examples.

What: Supervised fine-tuning is further training of a pretrained model on labelled prompt-and-answer pairs, adjusting its weights, fully or through adapters such as LoRA, so it reproduces the desired behaviour.

When: Task fine-tuning of language models took hold in 2018. Instruction tuning followed in 2021, and InstructGPT set the SFT-then-RLHF pipeline in March 2022. OpenAI's self-serve platform closes to new jobs on January 6, 2027.

Where: SFT sits between pretraining and preference or reinforcement learning in the post-training pipeline, and runs on lab clusters, cloud platforms such as Vertex AI, and fine-tuning APIs.

Why: Base models continue text rather than follow instructions. SFT is the cheapest reliable way to teach format, tone and narrow tasks, though it imitates rather than generalises and is increasingly paired with, or replaced by, reinforcement learning.