LLM-generated verse is formally stuck on rhyme and regular meter

As in work by @mellymeldubs.bsky.social et al, I examine formal tendencies in the poetic output of 9 LLMs. I find that LLM verse is “formally stuck” on rhyme & regular meter, producing these forms far more than even the formally strictest periods of literary history—and even when instructed not to.

Figure 1.Frequency of rhymed poems in the Chadwyck-Healey corpora compared with LLM-generated verse. Generative models were prompted for three types of poems: rhyming poems, unrhyming poems, and poems without specifying whether to rhyme. Points indicate mean likelihood; size indicates the number of poems per data point; whiskers show standard error.  The left-hand portion of the graph shows that historically, across the 1,000 sampled poems per half-century in the Chadwyck-Healey poetry collections, the average likelihood for a poem to rhyme descends over time, particularly since the late nineteenth century. Prior to that, the historical incidence of rhymed verse hovers between 71% and 90% for three centuries, before turning away from it in the late nineteenth century (71%) and plummeting in the early twentieth (15%)—with ultimately only 7% of poems written by postwar poets in rhyme.  Meanwhile, the right-hand portion of the graph, a distant reading of rhyme not in historical but generative verse, shows several striking findings. On average across the LLMs, prompts for a rhyming poem (shown in blue) and prompts simply for a poem without specifying whether to rhyme (shown in green) produce a body of artificial verse composed of 95% rhyming poetry. Not only is this incidence more than in any historical period of actual human-authored verse, but its similarity across these two categories of prompts suggests a remarkably stubborn association between the genre of poetry and the form of rhyme within these models. Furthermore, whether prompted simply to write a poem, a rhyming poem, or an unrhyming poem (shown in purple), generative models—despite the overwhelming bias in their training data toward the contemporary—rhyme more often than poets born in the late twentieth century, and sometimes significantly more. Finally, even prompting the models for an unrhyming poem yields rhyming poems 50% of the time on average across the models, more than seven times as often as in the …

Figure 6.Frequency of syllable stress per syllable position into 10-syllable lines, drawn from historical and generative sonnets. Points indicate mean likelihood; whiskers indicate standard error.  Even before attempting to metrically scan these syllable stresses, Figure 6 shows how generative sonnets arrange their unstressed and stressed syllables more regularly than in any known century of the form. That LLM imitations of Shakespeare’s sonnets (in red) dip much lower in syllable stress likelihood in the odd-numbered syllables and peak much higher in the even-numbered ones—more than any other sonnet source, including Shakespeare’s own—shows how inhumanly rigidly its syllable stresses adhere to an iambic pentameter metrical template. The second syllable in the line, for example, is stressed 62% of the time in twentieth-century sonnets; 63% Shakespeare; and 69% in pre-twentieth-century sonnets, partly given the more frequent use of traditional metrical variations like the trochaic inversion. But LLM verse stresses this syllable a striking 89% of the time, far exceeding the sonnets of any historical period from the seventeenth through the twentieth centuries. Although this syllable shows the largest difference between generative and historical sonnet meter, and although all sonnet sources rise in regularity toward the end of the line (a well-known phenomenon in metrics), generative sonnets maintain an historically unprecedented regularity in syllable stress across the line.

Part of Generative Formalism.

Originally posted on Bluesky.

← All notes