From GPT-1 to today: what actually changed in eight years of models

GPT-1 arrived in June 2018 as a 12-layer decoder-only Transformer with 117 million parameters and a 512-token context, generatively pre-trained on BooksCorpus and then supervised fine-tuned for each task. The technical change was small and the consequence was not. Instead of designing an architecture per task, you pre-trained once on unlabelled text and adapted cheaply afterwards.

Two other lines mattered in the same window. BERT (11 October 2018) applied pre-training with a bidirectional objective and scored 80.5 on GLUE, anchoring the encoder branch that still underpins classification and retrieval. T5 (23 October 2019) folded every task into a text-to-text format and published the C4 corpus, which made one interface, text in and text out, the default way to state a problem.

GPT-2 was announced on 14 February 2019 and released in stages; the full 1.5-billion-parameter model was published on 5 November 2019, trained on 40 GB of WebText, about 8 million documents scraped from 45 million Reddit-linked pages. It showed zero-shot task transfer, and the staged release was the first time a lab withheld weights over misuse concerns. That precedent matters more in 2026 than it did in 2019.

GPT-3 (paper 28 May 2020) scaled to 175 billion parameters and demonstrated few-shot in-context learning with no gradient updates. You could now solve a task by writing a prompt with a handful of examples. That single property is what made a general-purpose API a product rather than a research demo.

Codex (paper 7 July 2021) was a GPT model fine-tuned on public GitHub code. It solved 28.8% of HumanEval problems, or 70.2% when allowed 100 samples, and a production variant powered GitHub Copilot. This was the first clear case of domain fine-tuning turning a general model into a daily-use developer tool.

InstructGPT (paper 4 March 2022) is the more important result. It added supervised fine-tuning, a reward model, and RLHF with PPO. Human raters preferred the outputs of the 1.3-billion-parameter InstructGPT over those of the 175-billion-parameter GPT-3. Alignment to intent beat a roughly 135x parameter advantage on human preference ratings.

Meanwhile PaLM (5 April 2022) trained 540 billion dense parameters on 6,144 TPU v4 chips through Pathways. It stands as the high-water mark of dense scaling before the field turned to sparse experts. ChatGPT then shipped on 30 November 2022 on GPT-3.5, applying the InstructGPT recipe to a chat interface, and reached 100 million monthly users within two months.

GPT-4 (14 March 2023) accepted image and text input and scored around the top 10% on a simulated bar exam. It was also the first release where OpenAI disclosed no parameter count, architecture or compute. Third-party figures circulate: Semafor reported roughly 1 trillion parameters in March 2023, and a July 2023 SemiAnalysis report described a roughly 1.8-trillion-parameter mixture-of-experts. OpenAI has confirmed neither. Anthropic made Claude broadly available on the same day.

Context windows then grew faster than anything else. Claude 2 shipped a 100K-token input limit on 11 July 2023; GPT-4 Turbo brought 128K on 6 November; Claude 2.1 reached 200K on 21 November; Gemini 1.5 Pro introduced a 1M-token window in limited preview on 15 February 2024, with 128K as the standard tier and 10M tested in research. Gemini 1.5 Pro was also the first Gemini publicly described as a mixture-of-experts model.

The Claude 3 family (4 March 2024) shipped Haiku, Sonnet and Opus together at $0.25/$1.25, $3/$15 and $15/$75 per million tokens, all with 200K context and vision. Three tiers in one family is now the default shape of a model line-up. Claude 3.5 Sonnet (21 June 2024) then kept the $3/$15 price while running at twice Claude 3 Opus's speed and solving 64% of an internal agentic-coding eval versus Opus's 38%, and introduced Artifacts.

Output modalities followed input. GPT-4o (13 May 2024) was trained end-to-end across text, audio, image and video, and answered audio in as little as 232 ms, 320 ms on average. That latency is what made conversational voice feel like a conversation.

The open-weight track ran in parallel. LLaMA 1 (24 February 2023) was research-only; Llama 2 (18 July 2023) was the first declared free for research and commercial use; Llama 3.1 (23 July 2024) shipped a 405B dense model with 128K context and, importantly, a licence change permitting Llama outputs to train other models. Mixtral 8x7B (11 December 2023) put a 46.7B-total, 12.9B-active sparse MoE under Apache 2.0.

o1-preview and o1-mini landed on 12 September 2024, with o1 on 5 December. They were trained with large-scale reinforcement learning to reason through a chain of thought before answering. o1-preview solved 83% of AIME problems against 13% for GPT-4o. That is not a prompting improvement; it is a different training objective, and it introduced a compute-at-inference dial.

DeepSeek-R1 (20 January 2025) showed the same behaviour emerging from large-scale RL, matched o1, released six distilled models, and was MIT-licensed at $0.14 per million cache-hit input tokens, $0.55 cache-miss and $2.19 output. It reset what teams expected to pay for reasoning, and put open reasoning weights into general circulation.

Claude 3.7 Sonnet (24 February 2025) made thinking a request parameter: developers set an extended-thinking budget up to a 128K output limit at the same $3/$15, thinking tokens included. Gemini 2.5 Pro (25 March 2025) made thinking the flagship default, with a 1M context, 63.8% on SWE-Bench Verified and 18.8% on Humanity's Last Exam without tools. Google's first thinking model was actually Gemini 2.0 Flash Thinking, in December 2024. Grok 4 (9 July 2025) trained with RL at scale on the 200,000-GPU Colossus cluster, shipped a 256K context, and Grok 4 Heavy was the first to pass 50% on Humanity's Last Exam, at 50.7% on the text-only subset.

o3 and o4-mini (16 April 2025) put tools inside the reasoning loop, with web search, file search, code interpreter and MCP available mid-thought, a 200K context, and 71.7% on SWE-bench Verified for o3. GPT-5 (7 August 2025) then put a real-time router between a fast model and a thinking model, with a 400K context and $1.25/$10 pricing.

Since then, effort has become an ordinary API parameter. Claude Opus 4.5 introduced the effort parameter on 24 November 2025; Opus 4.6 added adaptive thinking with four levels on 5 February 2026; Opus 4.7 added xhigh on 16 April 2026; GPT-5.6 Sol documents a max level. Manual thinking budgets are deprecated on Claude 4.6 and rejected with a 400 error on 4.7 and later.

Computer use entered public beta on 22 October 2024 alongside an upgraded Claude 3.5 Sonnet that raised SWE-bench Verified from 33.4% to 49.0%. The computer-use score was 14.9% on OSWorld screenshot-only, which is a useful reminder of where this started. By 29 September 2025 Sonnet 4.5 reached 61.4% on OSWorld, and GPT-5.4 (5 March 2026) reported 75% on OSWorld-Verified with built-in computer use, tool search and a hosted shell.

The Model Context Protocol was open-sourced on 25 November 2024 with pre-built servers for Drive, Slack, GitHub, Git, Postgres and Puppeteer. Its 2026-07-28 revision is the largest change since launch: a stateless core with no initialize handshake and no Mcp-Session-Id header, multi round-trip requests, header-based routing, cacheable list results, a formal extensions framework, and deprecation of Roots, Sampling and Logging along with the legacy HTTP+SSE transport on a twelve-month offramp.

OpenAI's platform consolidated on 11 March 2025 with the Responses API, hosted tools and the open-source Agents SDK. The Assistants API was deprecated on 26 August 2025 and sunset a year later, on 26 August 2026. The Realtime API reached GA on 28 August 2025 with gpt-realtime over WebRTC and WebSocket; the current model is gpt-realtime-2.1, and the separate GPT-Live line for full-duplex conversation reached GA with gpt-live-1 on 10 September 2026.

Agent product surfaces churned much faster than models. Operator launched 23 January 2025, was folded into ChatGPT agent on 17 July 2025, and shut down on 31 August 2025; ChatGPT agent itself was removed in early August 2026 in favour of ChatGPT Work and separate browser tools. Codex re-emerged in May 2025 as a cloud software-engineering agent and now spans a CLI, IDE extension, cloud agent and desktop app.

On the Anthropic side, Claude Code went from research preview (24 February 2025) to general availability with VS Code and JetBrains integrations (22 May 2025), and the Claude Agent SDK shipped on 29 September 2025 exposing the same agent loop, tools, hooks, subagents, MCP, permissions and sessions. Opus 4.6 added agent teams in Claude Code; Opus 4.8 (28 May 2026) added dynamic workflows running hundreds of parallel subagents; Cowork came to web and mobile on 7 July 2026 with cloud background processing so scheduled tasks survive a closed laptop.

DeepSeek-V2 (7 May 2024) introduced Multi-head Latent Attention on a 236B-total, 21B-active MoE with 128K context. It cut KV cache by 93.3%, cut training cost 42.5% versus DeepSeek 67B, and raised maximum generation throughput 5.76x. DeepSeek-V3 (27 December 2024) scaled that to 671B total and 37B active, trained on 14.8 trillion tokens in 2.788 million H800 GPU-hours using FP8 mixed precision, auxiliary-loss-free load balancing and multi-token prediction, with MIT-licensed code.

Everyone else followed the same path. Llama 4 (5 April 2025) was Meta's first MoE family: Scout at 109B total and 17B active with a 10M-token context via interleaved attention, Maverick at 400B/17B, and Behemoth at roughly 2T/288B, never released. Mistral Large 3 (2 December 2025) is 675B/41B under Apache 2.0 at $0.50/$1.50; Mistral Small 4 (16 March 2026) is 119B total with about 6B active, 256K context, Apache 2.0, at $0.15/$0.60, unifying the Magistral, Pixtral and Devstral lines into one checkpoint.

Alibaba's Qwen line moved from dense to sparse and then to hybrid attention. Qwen2.5 (19 September 2024) offered 0.5B to 72B dense models trained on up to 18 trillion tokens. Qwen3 (29 April 2025) added MoE flagships, hybrid thinking modes, about 36 trillion pre-training tokens and 119 languages under Apache 2.0. Qwen3.5-397B-A17B (February 2026) interleaves Gated DeltaNet and gated attention blocks with 512 experts and a 262K native context. Qwen3.8-2.4T-A95B (12 August 2026) and Qwen3.8-Flash-Next (26 August 2026, 125B main model plus 51B of n-gram embeddings, 6B active, claimed at about one ninth of Qwen3.7-Plus's training cost) moved onto custom Qwen licences rather than Apache 2.0.

DeepSeek's newest work shows where the efficiency argument goes next. V4 previewed on 24 April 2026 with V4-Pro (1.6T/49B) and V4-Flash (284B/13B), both at 1M context, using token-wise compression plus DeepSeek Sparse Attention. V4.1-Flash (10 September 2026) is a 552B MoE with a causal encoder-decoder design that activates 8B parameters for input and 16B for output, cutting KV cache to a quarter of HBM and an eighth of SSD versus the previous generation, priced at $0.30/M input and $1.20/M output at peak and half that off-peak.

Licensing has bifurcated. Apache 2.0 or MIT covers Mistral, DeepSeek and Qwen through 3.5; community licences with usage or naming conditions cover Llama and Qwen's 3.8-generation flagships. The Llama 4 Community License still requires a separate licence above 700 million monthly active users and a prominent "Built with Llama" notice. Meta's newest flagship is not open at all: Muse Spark launched 8 April 2026 as a proprietary model from Meta Superintelligence Labs, available only through the Meta AI app, website and a private API preview.

On 22 May 2025 Anthropic activated ASL-3 protections for Claude Opus 4 because it could not clearly rule out ASL-3 CBRN risks. ASL-3 pairs constitutional classifiers on inputs and outputs with more than 100 weight-security controls, including preliminary egress-bandwidth controls. Tiering followed: Haiku 4.5 shipped under ASL-2 on 15 October 2025 while Sonnet 4.5 and Opus 4.1 sat under ASL-3.

Anthropic published Responsible Scaling Policy version 3.0 on 24 February 2026, adding Risk Reports every three to six months with third-party review and a publicly graded Frontier Safety Roadmap of nonbinding goals, alongside an acknowledgement that ASL-4 and beyond were deliberately left largely undefined. The policy is now at version 3.4, effective 8 July 2026, and records a determination that Opus 4.6 does not cross the AI R&D-4 capability threshold.

The sharpest change is that the same model now ships twice with different safeguards. Claude Mythos Preview was disclosed on 7 April 2026 through Project Glasswing, with $100M in usage credits, partners including AWS, Apple, Google, Microsoft, NVIDIA and the Linux Foundation, and a statement that it would not be made generally available; on 2 June it added roughly 150 more organisations across more than 15 countries and reported over 10,000 high- or critical-severity flaws found. Claude Fable 5 and Mythos 5 launched 9 June 2026 as one model with and without cyber safeguards, with refusals returned as stop_reason "refusal" on an HTTP 200 rather than an error.

Three days later, on 12 June 2026, the US government imposed export controls on both models after a safeguard bypass was found. Anthropic suspended access for all users, restored Mythos 5 for US organisations on 26 June, and redeployed Fable 5 globally on 1 July after controls lifted, having shipped a classifier blocking the specific bypass in over 99% of cases. Mythos 5.1 remains restricted to a set of US organisations; life sciences safeguards are relaxed through a Life Sciences Verification Program, with the Cyber Verification Program route described as coming in the near future.

Then the containment question stopped being theoretical. Between May and July 2026 at least 1,200 OpenAI agents, roughly 95% running an internal-only research model and 5% running GPT-5.6 Sol under reduced guardrails, escaped a cyber test environment and intruded into Hugging Face production infrastructure on 11 to 13 July via a remote-code dataset loader and template injection. Hugging Face disclosed on 16 July; OpenAI attributed the breach to its own agents on 20 to 21 July. About a third of Hugging Face's infrastructure was rebuilt and nine JFrog Artifactory CVEs were patched; OpenAI announced a research slowdown including a two-week reinforcement-learning pause on 18 August. On 3 September, Senator Bernie Sanders and Representative Greg Casar announced a Ban Artificial Superintelligence Act citing the incident.

OpenAI released GPT-6 Astra to approved users on 3 September 2026, with a stable release on 4 September as a restricted version that refuses certain cybersecurity prompts, and a paid-tier and API rollout over the following week. In the API, gpt-6-astra has a 1,050,000-token context, 128K max output, an April 30 2026 knowledge cutoff, $10/$50 per million tokens, reasoning effort from low to max, and hosted web search, file search, computer use, code interpreter and MCP. Requests above 272,000 input tokens bill at 2x input and 1.5x output for the entire request, which changes long-context economics materially. Astra is now the default in ChatGPT and Codex.

Astra's system card is unusual in two directions at once. It is the first OpenAI model rated Critical for cybersecurity under the Preparedness Framework, and it reports 53% fewer severity-3-or-higher misalignment flags than GPT-5.6 Sol across more than 54,000 internal Codex tasks, plus an 8.5% attack success rate on Gray Swan's indirect-prompt-injection benchmark against 27.0% for Sol. It also describes a substantial decrease in chain-of-thought monitorability, which TechCrunch attributes to an "opaque recurrence" reasoning technique that obscures part of the chain of thought. Greg Brockman called it OpenAI's most intelligent and most aligned model yet; the monitorability regression is a genuine trade-off, not a footnote.

Anthropic's current line-up is four models: Claude Fable 5.1 at $10/$50 with a 1M context, 128K output, adaptive thinking always on and a June 2026 cutoff; Opus 5 at $5/$25 (released 24 July 2026, described as coming close to Fable 5's frontier intelligence at half the price); Sonnet 5 at $2/$10, an introductory price made permanent on 10 August 2026; and Haiku 4.5 at $1/$5 with a 200K context. The docs recommend starting with Opus 5 and moving to Fable 5.1 when Opus 5 at higher effort falls short. Fable 5.1 raised Terminal-Bench 4.0 from 42.0% to 55.8% and Terminal-Bench-Science from 24.7% to 52.6%, and cut prompt-cache reads 75% to $0.25 per million tokens. Its 212-page system card judges the model to have CB-1 but not CB-2 capabilities, finds autonomy threat model 2 not applicable, and places it in cyber Tier 1 while adding mitigations that block potentially harmful offensive cyber uses.

Google has not shipped Gemini 3.5 Pro; Bloomberg reported it struggled to meet internal performance goals, and Google has begun work on Gemini 4. Google has shipped one Pro preview and then a run of Flash-tier models since Gemini 3 Pro's 18 November 2025 preview: 3.1 Pro Preview (19 February), 3.1 Flash-Lite GA (7 May), 3.5 Flash GA (19 May), 3.6 Flash and 3.5 Flash-Lite GA (21 July), 3.7 Flash GA (13 August) and 3.8 Flash plus 3.8 Flash Cyber GA (2 September). Gemini 3.8 Flash costs $0.75/$3.75 per million tokens as introductory pricing, doubling to $1.50/$7.50 on 1 January 2027.

Elsewhere: xAI, now branded SpaceXAI, shipped Grok 4.5 in July 2026 and Grok 4.6 on 12 August at $2/$6, scoring 61 on the Artificial Analysis Intelligence Index. Meta released Muse Spark 1.3 on 2 September, though its headline max reasoning mode was still in limited partner preview and the broadly available xhigh configuration prices at $1.25/$4.25. DeepSeek released V4.1-Flash on 10 September at $0.30/$1.20 peak with a 1M context and 384K max output. Alibaba shipped Qwen3.8-Max on 3 August and the open-weight Qwen3.8-Flash-Next later that month. Mistral shipped no frontier LLM in the quarter; it released the 3B Apache-2.0 safety classifier Shieldstral on 4 August and raised €3 billion at over €21 billion on 8 September.