A poem about a haircut

The idea

Stanisław Lem’s The Cyberiad was published in Polish in 1965. It is a cycle of tales about two robot constructors, Trurl and Klapaucius, and the official Lem site names “the nature of creativity” among its themes (lem.pl). Michael Kandel’s English translation appeared in 1974 (The Encyclopedia of Science Fiction).

One story, “The First Sally (A), or Trurl’s Electronic Bard”, is about a machine that writes poetry. Trurl builds it to silence his rival “once and for all”. First he collects “eight hundred and twenty tons of books on cybernetics and twelve thousand tons of the finest poetry” and reads all of it. Then the real problem dawns on him. The program in a poet’s head was written by the poet’s civilisation, which was programmed by the one before it, back to the Dawn of Time. So to program a poetry machine, “one would first have to repeat the entire Universe from the beginning—or at least a good piece of it” (the passage is quoted at length in a Humanist Discussion Group post by Jan Rybicki). Trurl builds it anyway.

The scene that matters here is the test. Klapaucius looks for a request as difficult as possible. In Kandel’s translation, he is “aware that any argument on the quality of the verse the machine might be able to produce would be hard if not impossible to settle either way” (Wichary, 2015). So he asks for something anyone can check: a poem about a haircut, “lofty, noble, tragic, timeless”, in six rhymed lines, with “every word beginning with the letter s”. The machine answers at once: “Seduced, shaggy Samson snored.” Klapaucius raises the bar to a sonnet “all in g”.

Trurl objects. He did not build “a machine to solve ridiculous crossword puzzles! That’s hack work, not Great Art!” So Klapaucius asks for a poem “lyrical, pastoral, and expressed in the language of pure mathematics. Tensor algebra mainly”, and “with feeling”. Trurl balks: “Love and tensor algebra?” (De Smedt, 2013, quoting Kandel’s translation with the permission of the Lem estate). The bard obliges. What the poets of Trurl’s world make of all this is the rest of the story, and worth reading yourself.

The English poem is itself a solved constraint. In Lem’s Polish the alliteration was on C, and the request was a different one. Kandel wrote a new poem that met the same kind of constraint in English (Wichary, 2015). A human translator had to do, under the same kind of rule, what the fictional machine does instantly.

The bridge

Klapaucius splits evaluation in two. One part is constraints anyone can verify: six lines, a rhyme scheme, the first letter of every word. The other is quality: whether the poem is lofty, tragic and moving. He tests the first because he cannot settle the second. Research on machine creative writing is built on the same split, for the same reason.

Benchmark designers make the split explicit. IFEval tests “verifiable instructions” such as “write in more than 400 words”. The authors chose them because human evaluation is expensive, slow and not reproducible, and model judges can be biased (Zhou et al., 2023). POEMetric, a poetry benchmark presented at ICLR 2026, separates “basic instruction-following abilities” (form and theme) from “advanced creative abilities” such as creativity, idiosyncrasy and emotional resonance (Li et al., 2026). The haircut poem belongs in the first group. The tensor love poem, with feeling, belongs in the second.

Trurl’s objection also applies. Passing the crossword says little about the art. In the research below, models do well on the checkable layer and less well on the rest, and the rest turns out to be hard to measure at all.

The story’s premise maps onto training. Trurl reads twelve thousand tons of poetry before he builds anything, then decides the real program was written by all of history. Language models take the shortcut his reasoning rules out: they learn from the written record instead of re-running the world. How much of their output is new, and how much is reassembled from that record, is now something researchers measure.

What research says now

Form is the checkable layer, and it is not trivial

COLLIE builds constraint tasks from a grammar, at the level of words, sentences, paragraphs and passages. In its first evaluation, GPT-4 met 50.9% of constraints on average. Position mattered: starting a sentence with a given word succeeded 100% of the time, ending one with a given word 40% to 60% (Yao et al., 2023). Rhyme is a constraint on endings.

Letter constraints like Klapaucius’s are awkward for another reason. Most models process text as multi-character tokens, without direct access to the characters inside them. CUTE found that models seem to know how their tokens are spelled but fail to use that knowledge to manipulate text (Edman et al., 2024).

Rhyme, at least, gets planned. Anthropic’s interpretability team traced how Claude 3.5 Haiku writes a rhyming couplet. Before the second line starts, the model activates candidate end words, such as “rabbit” and “habit” for a line that has to rhyme with “grab it”, and then writes towards one of them. The team found planned-word features in about half of the poems they examined, and presents the results as specific examples, not a general claim (Lindsey et al., 2025).

POEMetric tests the whole form. It prompted 30 models with the form, rhyme, metre and theme of 203 human poems in seven fixed forms, 6,090 generated poems in total. With Gemini 2.5 Pro as judge, the best model scored 4.26 out of 5 on form accuracy and 4.99 on theme. Some weaker models fell back on a default, for instance the common ABAB rhyme scheme for ballads (Li et al., 2026).

Quality depends on who reads

Porter and Machery took five poems each from ten well-known poets, from Chaucer to Dorothea Lasky, and had ChatGPT 3.5 write five poems “in the style of” each, keeping the first five it produced. Non-expert readers identified the AI poems with 46.6% accuracy, below chance. They were more likely to judge the AI poems human than the human ones, and rated them higher on qualities such as rhythm and beauty. The authors’ explanation: the simpler AI poems are easier for non-experts to understand, and readers took the complexity of real poems for machine incoherence (Porter and Machery, 2024).

Expertise changes the verdict, and so does training. In a preregistered study, MFA-trained writers and three frontier models wrote excerpts of up to 450 words in the styles of 50 award-winning authors. With in-context prompting, 28 MFA-trained readers strongly disfavoured the AI text on quality (odds ratio 0.13), while 516 college-educated general readers favoured it (1.82). Fine-tuning ChatGPT on each author’s complete works reversed the expert verdict: MFA-trained readers now favoured the AI text on quality (1.87), general readers more strongly (5.42). The authors’ mediation analysis attributes the reversal to fine-tuning removing detectable AI quirks such as cliché density. The study is prose, not poetry, and the preprint is under review (Chakrabarty, Ginsburg and Dhillon, 2025).

Rubrics and machine judges

One answer to “hard to settle” is a rubric. The Torrance Test of Creative Writing turns creativity into 14 binary tests across fluency, flexibility, originality and elaboration. Ten creative writers applied it to 48 stories. Stories written by LLMs passed 3 to 10 times fewer tests than stories by professional authors, and none of the LLMs tried as assessors correlated positively with the experts (Chakrabarty et al., 2024).

Another answer is a trained judge. LitBench pairs stories from the r/WritingPrompts subreddit and labels the winner by upvotes, keeping only pairs with a clear margin. On its 2,480-pair test set, the best off-the-shelf judge, Claude 3.7 Sonnet, agreed with the labels 73% of the time; trained reward models reached 78% (Fein et al., 2025). The ground truth here is what a large forum upvoted, which is one definition of quality among several.

Judges bring their own reading history. POEMetric chose Gemini 2.5 Pro because it agreed with human experts more often than DeepSeek-R1 or GPT-4o (0.662 against 0.548 and 0.438). The same judge recognised 80 of the 203 human poems, 39.4%, by reciting the original or recognising the poet’s style (Li et al., 2026). Authorship was hidden from the judge, but in those 80 cases it already knew the answer.

A comparison of four creativity measures, presented at EACL 2026, tested perplexity, LLM-as-a-judge, n-gram overlap with the web, and syntactic templates. A metric that worked in one domain failed in another, and metrics often disagreed on the same texts. LLM judges gave inconsistent verdicts under minor prompt variations (L. Lu et al., 2025).

Novelty, measured against the library

Trurl’s twelve thousand tons of poetry has a modern counterpart: the web text models learn from. The Creativity Index measures how much of a text can be rebuilt from verbatim and near-verbatim snippets found on the web. Professional human authors scored on average 66.2% higher than LLMs, and alignment lowered the models’ scores by 30.1% on average (X. Lu et al., 2024). The EACL comparison above found that the index separates creative from uncreative texts in creative writing but not in problem-solving, and that it mainly measures lexical diversity (L. Lu et al., 2025).

Many machines, one voice

Infinity-Chat collects 26,000 real open-ended queries. Its authors found that a single model repeats itself and, more strongly, that different models give strikingly similar answers. They call it the Artificial Hivemind. Models, reward models and model judges were also less well calibrated to human ratings where annotators’ individual preferences diverged (Jiang et al., 2025).

Verbalized Sampling traces part of this to the preference data. Annotators systematically favour familiar text, and post-training on their choices narrows the output. Asking a model for several responses together with their probabilities raised diversity in creative writing by 1.6 to 2.1 times over direct prompting (Zhang et al., 2025).

The effect reaches human writers. In an online experiment, writers who got story ideas from an LLM wrote stories rated more creative, better written and more enjoyable, especially the less creative writers. Their stories were also more similar to each other (Doshi and Hauser, 2024).

Where the analogy breaks

Lem puts the difficulty in a different place. His bard meets the letter constraint instantly, and only taste is in doubt. The best model in POEMetric scores near-perfectly on theme, while letter-level and sentence-ending constraints are where COLLIE and CUTE found failures in their 2023 and 2024 tests. My read: Klapaucius’s checkable test is harder for token-based models than it looks, for a reason Lem had no cause to imagine.

Lem’s bard is one machine with one voice. The pressing problem now is many machines with nearly the same voice, as the Hivemind results show.

Lem can stipulate that the poems are great. Research cannot. The best evidence says the verdict depends on the reader, on the training and on the judge’s own reading history.

Trurl’s premise is a joke that works as a hypothesis. My read: the Creativity Index results fit the idea that models lean heavily on the written record. They do not show that a system without lived history cannot write a great poem. Speculation: as fine-tuning removes the quirks that detectors and experts use, the checkable layer will matter less, and evaluation will be left with Klapaucius’s problem, an argument about quality that is hard to settle.

Follow the thread

• POEMetric (Li et al., 2026): data and code are released, so you can run a new model on the seven fixed forms and compare the rule-based form check with the judge’s scores.
• COLLIE (Yao et al., 2023): the constraint grammar is built to be extended, so a Klapaucius constraint, such as every word starting with one letter, can be added and measured.
• Torrance Test of Creative Writing (Chakrabarty et al., 2024): 14 binary tests you can apply as a reader before you trust any automated score.
• LitBench (Fein et al., 2025): the trained reward models are released; check where an upvote-trained judge disagrees with an expert rubric.
• Infinity-Chat (Jiang et al., 2025): 25 human annotations per example make it possible to study where individual taste departs from the average.
• Verbalized Sampling (Zhang et al., 2025): a training-free prompt with public code; try it on poems and measure how much the outputs differ.
• AGC-Bench (Beaty et al., 2026): 78 datasets and a public leaderboard; on a human-matched subset, the authors report the top human still ahead of the top LLM.
• An open question with no paper attached: what would a Klapaucius test for value look like, one whose result two critics could not argue about?