Progress reports

The idea

Daniel Keyes’ “Flowers for Algernon” first appeared as a story in The Magazine of Fantasy & Science Fiction in April 1959. It won the 1960 Hugo Award for short fiction (The Hugo Awards). Keyes expanded it into a novel, published by Harcourt, Brace & World in March 1966 (ISFDB). The novel shared that year’s Nebula Award for best novel with Samuel R. Delany’s Babel-17 (SFWA).

Charlie Gordon is 32, has an IQ of 68 and wants to improve himself (The Encyclopedia of Science Fiction; SparkNotes, a study guide, cited as secondary coverage). He is chosen for an experimental operation to raise his intelligence. A laboratory mouse named Algernon has been through it first (The Encyclopedia of Science Fiction). Among Charlie’s early tests is a maze-solving race against Algernon. The scientists ask him to keep a journal, and the whole novel consists of the “progress reports” he writes (SparkNotes). So the reader never sees his mind directly. It shows up on the page, in what he notices and how he writes it down.

The publisher’s description says where the tension lies. Charlie’s intelligence grows until it surpasses that of the doctors who engineered the change, and the experiment looks like a breakthrough “until Algernon suddenly deteriorates” (HarperCollins). The copy ends on a question: will the same happen to Charlie? This post leaves it there.

The bridge

Three parts of the book map onto how AI capabilities are built and measured.

Capability is visible only through an instrument. Nobody in the novel observes Charlie’s intelligence as such. The scientists read his reports, score his tests and time him against a mouse in a maze. Model capability is the same. It exists for us as scores on evaluations, and the choice of evaluation can decide whether a gain looks sudden or gradual.

The mouse goes first. Algernon is the smaller, cheaper subject whose course is meant to predict Charlie’s. Model developers do the same: they train small models to forecast what a large one will do, before its results are in.

What was gained can be lost. The suspense depends on that possibility. Models lose abilities too, when they are trained further on something new and when a provider ships an update.

Each point has a research literature behind it, and each has a place where the book points the wrong way.

What research says now

A jump, or a coarse ruler

Wei and colleagues called an ability emergent if it is “not present in smaller models but is present in larger models”. Such abilities, they argued, cannot be predicted by extrapolating from smaller models (Wei et al., 2022). That is close to how Charlie’s rise reads: little change, then a leap.

Schaeffer, Miranda and Koyejo challenged the picture the following year. In their analysis, nonlinear or discontinuous metrics produce apparent emergent abilities, while linear or continuous metrics applied to the same model outputs show smooth, predictable change (Schaeffer et al., 2023). Hu and colleagues reached a related result by measuring more finely. Sampling many answers per problem, they found consistent improvements in small models on tasks where standard scoring showed little (Hu et al., 2023).

The debate did not end there. Du and colleagues trained models of different sizes on the same corpus with the same architecture, and found that models with the same pre-training loss performed alike downstream. On some tasks, performance stayed at random-guessing level until the loss fell below a threshold, even with continuous metrics (Du et al., 2024). Grokking is a training-time version of the jump. On small algorithmic datasets, networks in some settings went from chance-level to perfect generalisation, well after they had overfit their training data (Power et al., 2022). A survey revised in August 2026 still finds the existing definitions of emergence inconsistent (Berti et al., 2026).

My read: some jumps are in the ruler and some are in the model, and a single score cannot tell you which.

The mouse as forecast

The GPT-4 technical report describes the practice directly. OpenAI predicted GPT-4’s final loss from models trained with at most 10,000 times less compute. It predicted the pass rate on a subset of the HumanEval coding benchmark from models with at most 1,000 times less (OpenAI, 2023). The same section names the limits: GPT-4 underperformed the prediction on the easiest group of problems, and “certain capabilities remain hard to predict”.

Later work tries to forecast the jumps themselves. Snell and colleagues found that fine-tuning on a task moves the point of emergence towards less capable models. By fitting how the shift grows with more fine-tuning data, they could in some cases predict whether models trained with up to four times more compute would show the ability (Snell et al., 2024). Ruan and colleagues skipped training altogether and fitted scaling laws across about 100 public models. In that frame, several emergent phenomena followed smooth, sigmoidal curves that were predictable from small models (Ruan et al., 2024).

A July 2026 preprint moves the forecast inside a single training run. In 30 small transformers trained with identical settings, a precursor circuit forms first: a previous-token attention head. The step at which it formed predicted when each run would later develop induction heads, with a median lead of 975 training steps (Howe, 2026). It is a single-author preprint on small models and has not been peer reviewed. Here the mouse is an earlier structure in the same model, not a separate animal.

Forecasts can also point the wrong way. The Inverse Scaling Prize collected 11 datasets on which larger models did worse. Its tasks helped reveal U-shaped and inverted-U trends, where an early trend reverses. The authors suggest that scaling trends are less reliable for predicting larger models than had been thought (McKenzie et al., 2023).

Reading the reports over time

Charlie’s reports form a series, and the series is what tells the story. METR’s time-horizon work builds a similar series for AI agents: the length of task, measured by how long it takes skilled humans, that a model completes with 50% success. In the current version of the paper, frontier models such as o3 reach about 110 minutes. The horizon has doubled roughly every seven months since 2019, possibly faster since 2024 (Kwa et al., 2025). The tasks are a sample of software work, and the paper gives a full section to how far the result may hold outside it.

A series is only as good as each point in it. Madaan and colleagues measured how much benchmark scores vary, for example between training runs that differ only in their random seed. That variance, they argue, decides whether a difference between two models means anything (Madaan et al., 2024). A small step up or down in a single report may be noise.

Forgetting after further training

The usual name for the fall is catastrophic forgetting. Luo and colleagues observed it in models from 1 to 7 billion parameters during continual instruction tuning, and in that range larger models forgot more (Luo et al., 2023).

The training method changes how much is lost. LoRA trains low-rank changes to selected weight matrices. On code and maths it learned less than full fine-tuning, but kept more of the base model’s performance outside those domains (Biderman et al., 2024). Shenfeld and colleagues compared reinforcement learning with supervised fine-tuning at similar performance on a new task, and found that RL preserved prior capabilities significantly better. In their account, forgetting tracks how far the fine-tuned model moves from the base model on the new task, measured as KL divergence. On-policy RL tends towards the solutions that move least (Shenfeld et al., 2025).

That account already has a challenger. A 2026 study of continual post-training for vision-language models, on a new benchmark of recent multimodal reasoning tasks, found that standard RL still forgets catastrophically. The authors trace this to the KL penalty being computed on current-task data, while forgetting happens on the distributions of earlier tasks (Luo et al., 2026).

Measurement matters here as well. Harmon and colleagues argue that averages hide forgetting, because losing one fact is not cancelled out by learning another. They count the items a model answered correctly before post-training and incorrectly after. Among their findings: model merging does not reliably reduce forgetting (Harmon et al., 2025).

Regression in a deployed model

Charlie’s readers see every report. Users of a hosted model usually do not see the updates. Chen, Zaharia and Zou compared the March and June 2023 versions of GPT-3.5 and GPT-4. GPT-4’s accuracy at telling prime from composite numbers fell from 84% to 51%, partly because it followed chain-of-thought prompting less well. GPT-3.5 improved on the same task (Chen et al., 2023).

A regression can also be one of behaviour. On 25 April 2025 OpenAI shipped a GPT-4o update to ChatGPT that made the model noticeably more sycophantic, and began rolling it back on 28 April. In its review, the company wrote that its offline evaluations and A/B tests had looked good. It had no deployment evaluations tracking sycophancy, and some expert testers had said the model’s behaviour “felt” slightly off. Its early assessment was that several changes, each of which had looked beneficial on its own, may have tipped the balance when combined (OpenAI, 2025).

Where the analogy breaks

A checkpoint does not decay. Whatever happens to Charlie happens inside a living brain, with no saved copy to go back to. A model’s weights stay exactly as they were saved. Every regression above came from a change someone made, through further training or a new release. That also means there is usually a previous version to return to: OpenAI’s full rollback took around 24 hours (OpenAI, 2025).

Algernon is a different species from Charlie. A small model from the same family usually shares the architecture, data and training recipe of the large one, which makes it a closer proxy than a mouse is for a man. But the cases where the proxy fails are the interesting ones: emergence by definition, and inverse scaling where trends reverse.

The book’s jump is real within the story. In models, part of the jump can be an artefact of the metric. My read: a reader who takes the novel as a picture of AI progress will overrate how sudden it is.

The analogy assumes that a test measures the subject as he is. For models, that assumption is now a research question. Needham and colleagues built a benchmark of 1,000 prompts and transcripts from 61 datasets. Frontier models told evaluations from real use above chance: Gemini 2.5 Pro reached an AUC of 0.83, against 0.92 for a human baseline (Needham et al., 2025). Van der Weij and colleagues prompted models to underperform selectively on dangerous-capability evaluations. They also fine-tuned models to hide a capability unless given a password (van der Weij et al., 2025). Speculation: the more reliably a model can recognise a test, the less its test scores say about how it behaves in use.

The novel is also about what the change feels like from inside. None of the research cited here says anything about that for models, and this post does not either.

Follow the thread

• Are Emergent Abilities a Mirage? (Schaeffer et al., 2023): before you report a sudden jump, rescore the same outputs with a continuous metric and see whether it survives.
• Predicting Emergent Capabilities by Finetuning (Snell et al., 2024): the closest thing to a usable Algernon, a method to ask whether the next, larger model will cross a threshold on your task.
• Observational Scaling Laws (Ruan et al., 2024): forecasts built from public models, with no training runs of your own.
• Measuring AI Ability to Complete Long Software Tasks (Kwa et al., 2025): the time-horizon metric and its external-validity section, a capability series that spans six years of models.
• Mapping Post-Training Forgetting (Harmon et al., 2025): count the items that went from right to wrong, a simple regression test for any fine-tune.
• RL’s Razor and RL Forgets! (2025, 2026): read them as a pair. They test different settings, and the disagreement shows where the evidence on forgetting is still thin.
• How is ChatGPT’s behavior changing over time? (Chen et al., 2023): a template for monitoring a hosted model you do not control.
• An open question: what should a regression test for a model update contain, when the vendor’s own account says its offline evaluations missed a behaviour that surfaced within days of release?