Not training data but fine-tuning explains formal stuckness on rhyme in LLMs

I investigate whether training data can explain this effect, and find some evidence that it cannot. Instead, I find that “instruction-tuning” and Reinforcement Learning Human Feedback—designed to produce “helpful” chatbots—may instill conservative, “idealizing” aesthetic tendencies in the models.

Figure 5.Frequency of rhymed verse in poems detected in open and closed LLM training data. Points indicate mean likelihood; size indicates the number of poems per data point; whiskers show standard error.  For comparison, I also reproduce similar results from Walsh, Preus, and Antoniak, who detect poems in both open and closed models by the identical procedure, in their case by querying for a more canonical sample from the Poetry Foundation (PF) and Academy of American Poets (AAP). Then, for all four cases (i.e., Chadwyck-Healey in open and closed datasets; PF/AAP in open and closed datasets), I measure rhyme in the poems discovered by these methods to be present or absent in the training data (Figure 5). Across all methods and datasets, the distribution of rhyme in poems found in LLM training data (shown in blue) is not disproportionately high compared to poems not found in that data (shown in red). A comparison of means between found and unfound poems’ rhyming shows in all cases either statistically insignificant differences or differences with small effect sizes. Moreover, though very few poems from the historical corpora used in this study were found in open LLM training data, those found even rhymed less often than those not found. These results suggest that LLMs’ tendency to overuse rhyme cannot be explained simply by an overrepresentation of rhyming poetry in their training data. Rather, this data points to something more fundamental in how these models process and generate poetic forms.

Figure 9.Frequency of rhymed verse across generative completions of historical poems, one “instruction-tuned” and the other a raw next token generator. Points indicate mean likelihood; size indicates the number of poems per data point; whiskers show standard error.  Figure 9 above shows that llama3.1:instruct rhymes far more often than its identically-trained llama3.1:text. Comparing the frequency of rhymed verse in these two variants of the same model shows that the raw text completion variant rhymes far less (with a statistically significant difference of means across all historical periods of the completed poems) than the one tuned to respond helpfully to user instructions. Does this helpful responsiveness push them, by accident, into more idealized aesthetic forms like rhythm and regular rhythm?

Finally, I speculate whether generative AI secretly encodes an aesthetics of its own—as if internalizing Cicero’s goals for art to “delight” & “instruct” to an obsessive, historically unprecedented degree. I propose “generative formalism” as a method for studying this hyper-historical aesthetics.

Most of all, I recommend you check out all the incredible articles in this special issue (culturalanalytics.org/issue/12788), with work by Tess McNulty, Laura Chapot, @jeddobson.bsky.social, Alexandre Miller et al, and @katelelkins.bsky.social.

Part of Generative Formalism.

Originally posted on Bluesky.

← All notes