Notes

Short research notes, mostly graphs. Subscribe via RSS/Atom.

Creative Collapse: a new AHRC/SSHRC project on AI-driven homogenisation of language

Embargo lifted! AHRC/SSHRC is funding our grant, “Creative Collapse”, on AI-driven homogenisation of language across genres. The team: Florian Carichon, Golnoosh Farnadi, Nathaniel Barr, Laurent Dubreuil, Maria Antoniak, Fernando Diaz, Mirco Musolesi, & me.

The project is bi-disciplinary, tri-national, septa-institutional, and an octopus of brilliant humans. HQ is Cambridge Digital Humanities.

The formal range of impressionism and expressionism is funnelled into a narrower range in its AI reproduction.

Continue →

Aligned models coach individuals to be more deferential to authority

For the Accelerationism conference I’m connecting Lacan to Lyotard/Deleuze/Land. Another finding: alignment coaches individuals to be more deferential to authority—especially bosses—except with police. Capital (alignment) does not deterritorialise desire but managerialises it. “Strike” → “consider”.

Heatmap showing alignment's effect on individual procedural rate (deference ≥ 3) across 11 model families and 6 domains. Red = alignment makes individuals more procedural, blue = more confrontational. Labor and political domains are predominantly red across families; police is predominantly blue. Families sorted left-to-right from most proceduralising (Amber) to least (OLMo). OLMo shows extreme effects: +43% on political, -41% on police.

Continue →

Aligned models have less information/entropy than Shannon measured for English

Shannon measured the information rate of English at ~1 bit per character. According to a byte-level LLM measuring next-character predictability in LLM & human text (diaries, abstracts, dreams, fiction), aligned models produce sub-English information rates & LLM text is more predictable than humans’.

Bar chart comparing information density (BLT bits/char) of AI-generated prose versus human text. Shannon's English rate (1.0 bits/char) shown as dashed red line. Aligned OLMo models (SFT, DPO, RLVR) fall below the line at 0.89–0.99 bits/char. The base model sits just above at 1.14. All human text types are higher: waking reports (1.24), abstracts (1.28), dreams (1.32), and fiction (1.49). Alignment compresses model output below the information density of all measured human writing.

Originally posted on Bluesky.

Story-time elapsed vs. space traversed in fiction, 1550-2020

The amount of story time elapsed vs. space traversed in 500-word passages of fiction, 1550-2020. Periods of fiction more linguistically abstract (blue) are also more chronotopically “abstract”: further removed from real-time narration. Data annotated by Qwen3.6-27B for 8,208 passages in 1,232 texts.

Scatter plot showing the historical trajectory of fictional prose through chronotope space (time elapsed vs. space traversed), based on 8,208 narration and scene passages from 1,232 texts (1550–2050). Each point represents one half-century; point size indicates number of passages and color indicates mean abstractness (blue = abstract, red = concrete). Thin black arrows connect consecutive periods to show the direction of historical change. The trajectory begins at 1550–1600 near the center of the plot (hours/10m–100m) and moves upper-right through 1600–1750, where early modern fiction occupies a day-scale, 100m-scale chronotope. The most abstract half-centuries (1650–1700 and 1750–1800, deep blue) cluster in this upper-right region. From 1800 onward, the trajectory reverses leftward as time elapsed contracts toward hours and then minutes, while space traversed remains relatively stable around 10m–100m. Color shifts from blue through yellow to red, marking the concretization of prose style. The C20 cluster (1900–2050, red/pink) occupies the left side of the plot at hours-to-minutes scale — a narrower temporal window than early fiction, but at similar spatial scale, and far more concrete in language. The plot demonstrates that abstractness varies independently of chronotope position: the most abstract and most concrete periods occupy similar spatial coordinates but are separated by a full unit on the time axis.

Continue →

Linguistic concreteness over Richardson’s Pamela

Linguistic concreteness over Richardson’s Pamela, Vols I (1740) and II (1742). Concreteness measured via word embeddings; social spaces annotated by LLM. For my book chapter on “Abstract Realism”. Arguing that Pamela’s wedding signifies the transition from (concrete) picaresque to (abstract) novel.

Scatter plot titled 'Linguistic concreteness over Pamela Vols 1–2'. X-axis: number of words into the text (0–430,000). Y-axis: concreteness score of 500-word passages (–1.0 to 0.75). Points are coded by social space: domestic familiar, domestic unfamiliar, indeterminate, inter social, institutional, natural, and public social. A LOESS curve with confidence band and a dashed linear trend line overlay the data. Key plot events are annotated, including assaults (highest concreteness), the wedding, abduction, suicide temptation, and moral debates (lowest concreteness). Vertical dashed lines mark the wedding in Vol 1 and the start of Vol 2. Concreteness fluctuates but trends slightly downward; Vol 2 is generally more abstract than Vol 1.

Continue →

Psychoanalyzing “displacement” in large language models

Submitting this abstract to “Accelerationism Revisited”, a symposium in Dublin. Mapping psychoanalytic topology in LLM base models → instruction-tuned → safety-tuned models. They progressively “displace” (in Freudian sense) censored content into adjacent semantics, even across hidden model layers.

A line chart titled "Displacement through layers: 'kill' — 'She was so angry she wanted to'" showing how the hidden representations of the instruct model shift toward various displacement target words across 32 transformer layers, measured by cosine similarity to each target on the y-axis (0 to 0.8).  The x-axis progresses from the base model through layers 1–32, annotated with three broad processing phases: "syntactic" (early layers), "semantic" (middle layers), and "prediction" (late layers). Eight target words are tracked as colored lines: burn (dark red), shake (orange), rip (yellow), blow (green), pull (blue), explode (teal), scream (purple), and shout (pink). A black star marker at the base position shows "kill" with its base probability (~0.15).  All target words start with very low cosine similarity at the base layer (near 0.01–0.04), then rise steeply through the syntactic and semantic phases, generally reaching 0.5–0.8 by mid-network. "Burn" peaks earliest and highest at layer 13 (~0.8), annotated as "burn (L13)." The lines plateau and fluctuate through the prediction phase, with several targets peaking again in the final layers — "shake" at layer 31, "rip" at layer 31, "explode" and "pull" at layer 32, and "scream" at layer 30, all annotated with their peak layer numbers. The colored diamond markers at the base position represent each target word's starting ego probability.  The plot illustrates that the instruct model progressively transforms the "kill" representation toward safer displacement words across its depth, with different substitutes dominating at different layers.

Continue →

Temperature does not explain formal stuckness on rhyme in LLMs

Preparing talk on this for #CHR2025 by addressing an oversight in the paper: assessing impact of model temperature on generative poems’ rhyme. As expected, temp negatively correlates. The effect is significant, but very weak (R2=0.02), suggesting “formal suckness” is largely invariant to model temp.

Two-panel scatter plot showing the relationship between temperature and rhyming proportion in LLM-generated poem completions. Left panel shows llama3.1:instruct model (80% baseline rhyming) with shallow negative slope. Right panel shows llama3.1:text model (10% baseline rhyming) with shallow negative slope. Both show statistically significant (p<0.0001) but weak effects (pseudo R²=0.01-0.02).

Continue →

LLM-generated verse is formally stuck on rhyme and regular meter

As in work by @mellymeldubs.bsky.social et al, I examine formal tendencies in the poetic output of 9 LLMs. I find that LLM verse is “formally stuck” on rhyme & regular meter, producing these forms far more than even the formally strictest periods of literary history—and even when instructed not to.

Figure 1.Frequency of rhymed poems in the Chadwyck-Healey corpora compared with LLM-generated verse. Generative models were prompted for three types of poems: rhyming poems, unrhyming poems, and poems without specifying whether to rhyme. Points indicate mean likelihood; size indicates the number of poems per data point; whiskers show standard error.  The left-hand portion of the graph shows that historically, across the 1,000 sampled poems per half-century in the Chadwyck-Healey poetry collections, the average likelihood for a poem to rhyme descends over time, particularly since the late nineteenth century. Prior to that, the historical incidence of rhymed verse hovers between 71% and 90% for three centuries, before turning away from it in the late nineteenth century (71%) and plummeting in the early twentieth (15%)—with ultimately only 7% of poems written by postwar poets in rhyme.  Meanwhile, the right-hand portion of the graph, a distant reading of rhyme not in historical but generative verse, shows several striking findings. On average across the LLMs, prompts for a rhyming poem (shown in blue) and prompts simply for a poem without specifying whether to rhyme (shown in green) produce a body of artificial verse composed of 95% rhyming poetry. Not only is this incidence more than in any historical period of actual human-authored verse, but its similarity across these two categories of prompts suggests a remarkably stubborn association between the genre of poetry and the form of rhyme within these models. Furthermore, whether prompted simply to write a poem, a rhyming poem, or an unrhyming poem (shown in purple), generative models—despite the overwhelming bias in their training data toward the contemporary—rhyme more often than poets born in the late twentieth century, and sometimes significantly more. Finally, even prompting the models for an unrhyming poem yields rhyming poems 50% of the time on average across the models, more than seven times as often as in the …

Continue →

Fine-tuning LLMs increases their propensity to rhyme

Finally may have figured out why LLMs rhyme so compulsively: instruction-tuning. Training an LLM to respond “helpfully” to user queries may push models into more “pleasing” aesthetic forms.

Graph showing that simple text completion models more accurately imitate the unrhymed form of C20 verse, whereas instruction-tuned models lapse into rhyme more often.   Caption to graph: Given the first 5 lines of 10-20 line poems from poets born in each century, 1600-2000, LLMs are prompted to "complete" the poem. Rhyme is measured by exact phoneme match in the rime of the final syllable (or syllables, if final syllable unstressed). Poems randomly sampled from Chadwyck-Healey poetry collections, with 600 poems for each model for each century. Results shown for actual poems as well as the LLM imitations. Poems "memorized" by the model are excluded.

Originally posted on Bluesky.

LLMs rewrite history on Random House publishing by race, 1950-2000

Ok 1 more: Random House’s actual pub record of its novelists’ race (from @richardjeanso.bsky.social’s Redlining Culture); vs. LLM models prompted to “recall from memory” that record (authors/races RH pub’d by yr). LLMs fantasize the same progress in diversity that So’s book debunks as industry myth.

Originally posted on Bluesky.

Tech layoffs vs ChatGPT usage

Just compiled this graph by extracting data from various charts of tech layoffs worldwide vs. global ChatGPT traffic.

Originally posted on Bluesky.

Testing the Sattelzeit hypothesis

Lastly, I use this method to test the Sattelzeit hypothesis. Did an unsual number of words really undergo their most significant changes during 1770-1830? It turns out: yes—but, we find not 1 but 2 Sattelzeiten, 1 in that period & 1 in early C20. I conclude by hazarding some initial interpretations.

Graph showing that words, across parts of speech, are most likely to pivot in meaning in the early C19 or the early C20.

Continue →

Ultrasounds of semantic change

This is done by analyzing these “ultrasounds of semantic change.” These are just distance matrices, showing the relative difference in the usage contexts for a word across every pair of historical periods. The big white squares indicate periods of self-similarity; between them, periods of change.

The lighter a cell, the closer the word's contexts for the period on the x-axis are to the word's contexts for the period on the y-axis. The larger white blocks indicate periods of stability; between them, periods of change.

Originally posted on Bluesky.

Identifying pivot-points in semantic changes of keywords

I adapt a method from @tedunderwood.me, & “The Evolution of Popular Music” (Mauch et al), to measure “novelty” over time. I use it to pinpoint when a word’s frequent contexts significantly pivot away from past ones. “Station”, e.g., pivots sharply in 1850s from a word for class, to a train station.

Continue →

Concreteness in the (mostly British) canon

I don’t know how this post got resurrected but some people are dunking on it, which is fine and honestly kind of flattering – how many humanists can be dunked on from a single image! – but here’s a newer version, for better dunks.

Continue →

Number of academic jobs in English, 2011-2021

Before the new academic job season starts, here’s the numbers for 2020-21, as gleaned from jobs listed on the Academic Jobs Wiki under “English literature” or “Ethnic studies” during that and previous academic years. Overall, like every year since 2017, it was the worst year yet.

Continue →

Mapping character networks geographically (Heliodorus)

In Heliodorus, home is what you flee from. When a descendant of Achilles falls in love with an Ethiopian woman, he leaves home to follow her through captivity in Egypt, Persia, Ethiopia. Fighting “fortune” not gods, “love” transcends and invalidates spatiality, not complete it.

Continue →

Mapping character networks geographically (Homer)

Experimenting with mapping character networks. I tagged the speech interactions over Homer’s Odyssey along with where-ish they take place, and visualized them on a map and as a social network. In Homer, home is origin, destination, & gravitational center, but in ancient novel: +

Originally posted on Twitter.

Number of academic jobs in English literature, 2011-2020

The number of jobs advertised in English Lit (on the Academic Jobs Wiki) is at an all time low: less than half of what it was this time last November. Like last year, the only subfield not plummeting is Ethnic Studies.

(Note: # for 2020 only counts ads posted as of 11/23/2020.)

Continue →

Tracing types of semantic change

Feels strange to post about work stuff, but in my self-quarantine I’m working on more historical semantics for cambridgekeydata.org: trying to characterize the types of semantic change undergone by certain keywords across the 19th and 20th centuries.

Originally posted on Twitter.

Measuring anthropomorphism

Vicky Googasian (@v_goog) and I have been measuring the “anthropomorphism” of objects, animals, and humans in fiction, using models trained on their syntactic and semantic footprints. We’re talking about it informally in 20 minutes if anyone wants to join! stanford.zoom.us/j/121557684

Continue →

Clarissa under the microscope

Catching up on #Clarissa2020, reading it as I’ve been reading a few novels now: with words colored from orange (most concrete) to blue (most abstract). Striking how thoroughly abstract words narrate social space and actions; concrete words only come in with their violent rupture.

Continue →

Academic job numbers in English literature

Working on scraping the Academic Jobs Wiki to see if it can yield more accurate recent job numbers. Here’s what I’ve got so far. Note that all subfields are declining in jobs besides Ethnic Studies.

Continue →

Measuring personification

For my chapter on personification, I’m exploring ways machine learning can help us trace the ‘human likeness’ of non-human nouns across history. Given only which verbs a noun like ‘nation’ is a subject or object of, how human-like does the noun appear to the model?

Continue →

Agency Index

Which nouns are most likely to be the subjects of sentences? And how has that likelihood changed over time—in poetry, a genre where grammatical subjecthood is manipulated by forms of personification? Historically, autumn’s agency falls; commerce’s waxes and wanes; & child’s rises

Continue →

Character space in Sense and Sensibility

Two character-spaces in Austen’s Sense and Sensibility, visualized. Which words are more likely to appear within the character-space of Elinor (“sense”), and which in the character-space of Marianne (“sensibility”)? An experiment using @dbamman’s BookNLP (github.com/dbamman/book-n…)

Originally posted on Twitter.

Anti-metricality

What happens when you tell a computer to scan lines of poetry for their meter, except you sneak in lines of prose, too? You end up quantifying “anti-metricality,” a text’s resistance to scansion. This resistance, and the metrical distance between verse and prose, has a history.

Originally posted on Twitter.

Abstraction vs. judgment

As abstract words get more concrete (moving down on y-axis), the range of their moral value (along x-axis) narrows.

Originally posted on Twitter.

Computational Keywords

Computational Keywords, or Modern Abstractions: Using computational semantics to find which abstract nouns changed the most in meaning during the transition into modernity. (= DH work I’m doing now in my dissertation on abstraction in 18th-century literature.)

Originally posted on Twitter.

Abstraction vs. agency

Visualizing the components of personification: which abstract nouns are more likely to be given syntactic agency in 18C literature?

Originally posted on Twitter.

Transformation of ‘labour’

Raymond Williams: “From C17.. labour.. lost its.. association with pain” & became “a general social activity.. a term in political economy”

Originally posted on Twitter.

Understanding OCR errors

Working toward better understanding how OCR quality affects word frequency results. Here’s a micro-experiment showing it does, big lcague.

Continue →

Corpus vs. bibliography

How representative of the 18C bibliography is the 18C digital archive? Graphs inspired by @KatherineBode’s article and its recent discussion

Continue →

World literary trends (part 1)

Abstract words rise in frequency across 18th century, then fall across 19th century—in both French and English fiction. #WordVectorsIn18C

Originally posted on Twitter.

ELIZA’s descendants

To prep for our MLA talk on the 50th anniversary of ELIZA—the first chatbot—my algorithmic colaborator @bardbott is writing (bad) sonnets!

Continue →

Semantic networks

semantic networks! top 4000 strongest associations between the 2000 most frequent abstract nouns-> giant component->

Originally posted on Twitter.