Aligned models have less information/entropy than Shannon measured for English
Shannon measured the information rate of English at ~1 bit per character. According to a byte-level LLM measuring next-character predictability in LLM & human text (diaries, abstracts, dreams, fiction), aligned models produce sub-English information rates & LLM text is more predictable than humans’.

Part of Libidinal Language Models.
Originally posted on Bluesky.