Daejeon, July 27–31
The difference between poetry and song is so entrenched in contemporary culture, it barely registers as something of note. Popular music grew into a juggernaut of the entertainment industry with optimized production and dissemination loops, while poetry remains one of the most autonomous forms of art, far from the market and large audiences. As usual, the existing boundaries become evident at moments of transgression: e.g. when Bob Dylan, a singer, received the Nobel Prize in Literature, it caused a visible amount of distress in the reading community—there was not really a book to show up for the prize (Ellis-Petersen & Flood 2016). On the other side, poetry serves as a space for artistic consecration, a source of the highest literary value: “Lana Del Rey is not just a musician she's a poet” reads one of the YouTube comments under the Ultraviolence music video. Yet, when Del Rey publishes her own book of poetry Violet Bent Backwards Over the Grass (2022), the texts there do not resemble her song lyrics, but are instead full of literary sensibilities: free verse, tradition and genre signalling (use of the haiku), a very low presence of rhymed lines, the lyrical “I”.
In hindsight, all these distinctions and boundaries are more than surprising. Historically, poetry and song knew little difference, bound by music and performance. Many languages retain significant overlap between words for “poem” and words for “song”; the word “lyrical” originates in Ancient Greek, meaning “accompanied by a lyre”. Many poetic terms are still strongly associated with song: “couplet”, “hymn”, “ode”, “sonnet”, “canto”. Romanticism, hand in hand with European national revivals, reintegrated song and ballad into Western poetics; the forged link between a vernacular language, song, and nation is the reason why national poets are called “bards” until the present day (Trumpener 1997). The paradox of maintaining strong social distinctions within something that has shared origins and shared form was highlighted by James Fenton who defended the performative aspect in poetry: “The taste that delighted in the rhythms of rap belonged to the same owner as the taste that had banished metre from poetry” (Fenton 2003: 12-13).
In this study, we want to contrast histories of poetry and song lyrics to see how the increased social difference between “words with music” and “words without music” is reflected in language and poetic form. In a broader sense we ask how poetry changed in its modern quest for autonomy, and stopped being a song. To do that we compare historical datasets of poetry (ProQuest) and song lyrics (Genius) across the dimensions of form (meter, line, and rhyme) and general language (use of parts of speech). Through the combination of similarity measures, bootstrapping, and predictive modeling via logistic regression, we show how poems and song lyrics accumulated differences over the 20th century, with prototypical song turning into a lament, or address, while prototypical poem developed nominative catalogue-like style based on open form, absence of rhyme, and low presence of personal pronouns.
Data. We use ProQuest poetry dataset (Piper 2018: 203) of canonical English poetry, filtered to the 18th-20th centuries by the birthdates of authors. Because of processing limitations, we sample 1000 poems per decade, which results in a subset of ~21,000 texts (7M tokens).
There is no existing dataset of historical popular / urban song, and many studies of modern popular music use Billboard-100 data, which began in the 1950s. To push this date further into the past, we turned to Genius, a platform for social annotation; the dataset holds over 5M song lyrics (Nayak 2021). Again, we turn to time-stratified sampling, using Genius’ internal metric of popularity (“hits”) as probability of including a text in our subset. This allows us to filter out most irrelevant entries, while having a sample that reflects modern cultural attention to popular song history. We limit records by the year 2000 to match the span of the poetry corpus. The final subset includes ~10,000 texts (1.5M tokens).
All texts were processed with udpipe for morphological annotation, RhymeTagger for rhyme detection (Plecháč 2018), and Prosodic Python module for scansion (Heuser 2022).
Features. We compare two corpora in the triangle of relationships: time, form, language. The form is represented by three measures: 1) regularity of accentual rhythm, 2) regularity of line length in syllables, and 3) proportion of rhyming lines in a poem. The first measure is designed to provide a single metric of how “organized” the rhythm of a given text is (Šeļa & Gronas 2022): it counts the intervals of unstressed syllables and measures their diversity (using Shannon’s entropy). To transform the measure to be more intuitively read as “regularity”, we take the inverse of entropy. We use a similar approach to counting line syllable lengths.
We use a simple part-of-speech approach to model language because we want to avoid capturing obvious thematic / lexical differences between the two corpora. At the same time, parts of speech are often very indicative of genre and register of a poem (Gasparov & Skulatcheva 2003) and can reflect some dimensions of narrativity (Bamman et al. 2025).
Methods. We combine all features (morphology and form) to represent each individual text. We rely on bootstrapped sampling and averaging over a timeline in a sliding window to map uncertainty in our approach. We rely on Jensen-Shannon divergence to map the development of two traditions relative to some point in the past (the 19th century); then we build a Bayesian logistic regression model that is trained to distinguish song and poetry at specific time slices.
Figure 1. Change in form over time in poetry (green) and song lyrics (orange). Shaded areas reflect .95 CI.
The largest shift in 20th-century modern poetry was the fall of regular meter and rhyme and the rise of free verse as the dominant form. Figure 1 shows the transition across all measurements for poetry. Both meter and rhyme for centuries were at the center of the definition of poetry and poetic speech, entangled with the national discourse and literary histories (Martin 2012; Levine 2024). The fall of the form meant a redefinition of what poetry is, and the new definition is used until today.
We see that this is not the case for song: it continues to be more regular rhythmically (if not being metrical) and maintains high use of rhyme. Only line length does not show the difference: in the presence of melody, the isosyllabicity of text segmentation becomes irrelevant.
Two lines on the left panel of Figure 2 trace the difference between song and poetry from the fixed corpus in the past (19th-century poetry). The rising orange line shows how general language use diverged dramatically in song lyrics, yet poetry maintained a surprising (given the “revolutionary” fall of the form) continuity with 19th-century literary conventions and affordances, and only slowly accumulating change.
Figure 2. Left: Jensen-Shannon divergence to 19th-century poetry (POS-tags only). Right: accuracy from logistic regression, 300 texts per sample, 10 samples per window, 5-fold CV.
Sliding logistic regression on the right side of Figure 2 demonstrates the same rising distinction, but synchronously: the model becomes better at distinguishing poems from song lyrics the further we go into the 20th century. Figure 3 shows the effects of predictors (model trained on the full dataset) alongside one of the most prototypical songs according to the classifier (Diana by Paul Anka), full of rhymes, interjections, pronouns, and auxiliary verbs. Conversely, one of the most typical poems is Lavinia Greenshaw’s verbless, rhymeless list of scenes with an unclear, hidden subject of the speech: “blue earth / black water / purple fields…”
Figure 3. Posterior effects of a single model built on post-1940s texts.
The shift towards free verse in English poetry was accompanied by anxieties over lost audiences and rising literary autonomy (Glaser 2020). We see how, over the course of the 20th century, poetry freed itself of things that defined it: not only giving up on form, but also on general lyric voice (White 2014)—the “I” that addresses the other and the world in the present. It also meant that poetry lost its song-like features, locking itself in a perpetual avant-garde state and reinforcing the field boundaries with formal and linguistic distinctions.
If we look past all the differences and conditions of text production, it can be said that song became a popular form of poetry, retaining and developing its abandoned devices (rhyming), serving as a medium for any spoken word of the vernacular. And while song lyrics are now far away from the core definition of poetry, modern poetry is far away from the popular definition of itself: non-expert readers persistently associate formal regularity with human hand, treating LLM-generated verse as “more human than human” (Porter & Machery 2024), while chatbot models keep producing “formally stuck” poetry (Heuser 2025), amplifying popular aesthetics based on some distorted version of poetry’s past.