Daejeon, July 27–31
This project, undertaken as an experiment in collaborative student-centered literary-computational inquiry, examines how four contemporary generative AI (genAI) models attempt to write poems in the form of the duplex. The duplex is a poetic form created by Jericho Brown in 2019, in his Pulitzer Prize-winning collection of poems, The Tradition, which blends together the themes and formal features of the sonnet, the pantoum, the ghazal, and the blues poem. Because of the form's many defining features, the small number of direct examples (Brown has authored only five), and the deeply personal themes of memory, grief, intimacy, and survival that shape the original poems, the question of whether, and to what degree, genAI can approximate the duplex form offers a rich site for computational-humanistic inquiry. We document our methods for eliciting meaning from Brown's duplex poems and explore questions about voice, authorship, creative production, and the value of poetry prompted by our genAI experiments. Finally, we reflect on conducting computational-humanistic inquiry in an undergraduate classroom, suggesting a more substantive approach to acquiring AI literacy.
In a pair of papers, Melanie Walsh, Anna Preus, and Maria Antoniak's "Sonnet or Not, Bot?" (EMNLP 2024) and Melanie Walsh, Anna Preus, and Elizabeth Gronski’s "Does ChatGPT Have a Poetic Style?" (CHR 2024) explore the capabilities of contemporary genAI models to recognize common poetic forms and create new poems in those same forms. They found that models could mimic forms superficially—for example, producing exactly 14 lines for a sonnet, but using language that was more uniform and constrained than human-authored poetry. More recently, in "Generative Aesthetics: On Formal Stuckness in AI Verse" (2025), Ryan Heuser observed that when prompted, genAI models often "demonstrate extreme formal stuckness, appearing unable to escape rhyme regardless of the prompt." These findings motivated us to test how these same (or similar) models might respond to the duplex, with its more complex set of features, both as a classification and language generation task.
Brown created the duplex by studying the sonnet and blending certain of its elements with the blues poem, the pantoum, and the ghazal. The result is a fourteen-line structure consisting of seven couplets. Each line has nine to eleven syllables, and the lines are linked through repetition: the second line of the first couplet becomes the first line of the second; the second line of the second becomes the first of the third; and continues up to the final line, when the poem returns to its opening line, the meaning wholly transformed. In a blog post on the Poetry Foundation website, Brown describes the duplex as a "house with two addresses," a form that holds different traditions while also creating room for surprise ("Invention," 2019). The way the poem repeats and reworks its lines matches Brown's thematic interest in how memories and feelings continually return, but never in quite the same way. The repetitive structure mirrors the heavy physical and emotional burdens that his speakers carry, suggesting that survival isn't linear but circular, involving continuous processes of remembering and redefining.
Our experiments depend on two datasets: human-authored poetry and AI-generated duplex poems. For the human-authored dataset, we began with the public domain dataset from Walsh et al.'s "Sonnet or Not" paper, removing all poetic forms other than the sonnet, pantoum, ghazal, and blues poem. We manually added examples of ghazals and blues poems from the Poetry Foundation and American Academy of Poets websites, plus Brown's five duplex poems.
We created AI-generated duplexes using four models: GPT4-32K, GPT4o-mini, Sonnet 4.5, and OLMo 3-7B-Instruct, selected for popularity and ease of access. We included OLMo because it claims to not include copyrighted work (Brown's poems are under copyright).
To create the poems, we used three prompt templates—"general," "figurative," and "specific"—documented by Walsh et al. in "Does ChatGPT Have a Poetic Style?", modified for duplex-specific themes. We determined themes using tags associated with two duplex poems on the Poetry Foundation website, supplemented by a thematic analysis of all duplexes conducted as a class.
We created parallel templates including Brown's "rules of the duplex" in his own words, resulting in six prompts for each of 48 themes, generating 1,019 total poems (slightly lower than expected due to OLMo's occasional non-responses).
Our experiments consisted of three main parts. First, we tested for memorization. Then, we tested whether our models could identify the duplex form, following methods documented in "Sonnet or Not?" Finally, we conducted a detailed analysis of the AI-generated duplexes using five distinct methods: two adapted from "Poetic Style" and three we designed ourselves. These methods are: 1) Monroe et al.'s "fightin' words" algorithm (2017) to determine top 10 words most associated with each model's duplexes; 2) Normalized pronoun counts for each model's duplexes; 3) Original measures for calculating repeated phrases within duplexes to observe whether models follow Brown's rules; 4) Cosine similarity of AI-generated poems to original duplexes from The Tradition; and 5) Comparison of syllables of model-generated poems to Brown's original duplexes. Code is available at: https://github.com/emory-qtm/genai-duplexes
Prior work has used prompting to discover whether texts exist in a model's training data. When we tried this approach, both GPT models stated they could not provide the next line because it contained copyrighted material. However, GPT4 offered a line-by-line interpretation, implying the model has Brown's poems in its training data. OLMo claims to exclude copyrighted material from training data.
Following the template documented in "Sonnet or Not?," we examined whether our four models could accurately classify the four poetic forms combined in the duplex, plus the duplex itself:
Table 1. F1/Accuracy Scores Across Models
| Model | GPT-4 | GPT-4o Mini | OLMo 3 | Claude 4.5 |
| Sonnet (F1) | .85 | .97 | .11 | .98 |
| Ghazal (F1) | .61 | .90 | .38 | .96 |
| Blues Poem (F1) | .67 | .77 | .54 | .90 |
| Duplex (F1) | .44 | .57 | .25 | .91 |
| Overall Accuracy | 0.61 | 0.89 | 0.33 | 0.96 |
These findings confirm Walsh et al.'s findings that poems with fixed formal features, such as sonnets and ghazals, are easier for models to classify. OLMo's poor performance supports its claims about excluding copyrighted material.
Using "Fighting Words," we determined the most significant words for each AI-generated poem set:
Table 2. Top Ten Most Significant Words per Model
| Original | GPT-4 | GPT-4o Mini | OLMo 3 | Claude 4.5 |
| love, is, home, leave, me, need, be, who, was, my | we, beneath, shadows, yet, their, still, weight, silence, it, what | find, hearts, beneath, their, through, echoes, night, laughter, dance, silence | holds, heart, through, echoes, silence, night, never, yet, still, we | never, their, her, still, while, each, through, that, what, we |
Brown's most significant words are general, while the AI-generated poems repeatedly use specific words like "silence" and "echoes." The use of auditory words confirms Walsh et al.'s findings.
We compared the average normalized pronoun usage of Brown's duplexes to the four AI-generated sets, confirming Walsh et al.'s findings about overuse of first person plural in AI-generated poetry.
Figure 1. Average normalized pronoun usage: Brown’s Duplexes vs. Four Models
We devised seven measures to quantify the various rules of repetition that the duplex entails. "Phrase echo strength" reflects the average length of repeated phrases (in consecutive words) from line to line. "Reframing repetition" reflects phrase repetition across the entire poem. "Echo contrast" is the difference between phrase echo strength and reframing repetition. "Word echo strength" indicates average word matches throughout the poem. "Word reframing similarity" indicates average matches for reframing pairs specifically. "First/last echo" measures compare first and last lines:
Table 3: Repetition Metrics Across Models
| Metric | Original | GPT-4 | GPT-4o Mini | OLMo 3 | Claude 4.5 |
| Phrase Echo Strength | 3.57 | 7.91 | 3.07 | 2.93 | 6.94 |
| Reframing Repetition | 0.63 | 0.44 | 0.72 | 2.0 | 0.98 |
| Echo Contrast | 2.94 | 7.47 | 2.34 | 0.93 | 5.95 |
| Word Echo Strength | 3.09 | 5.14 | 1.80 | 1.84 | 4.47 |
| Word Reframing Similarity | 0.4 | 0.13 | 0.31 | 1.14 | 0.52 |
| First/last Echo Phrase | 5.20 | 8.23 | 5.23 | 4.88 | 7.89 |
| First/last Echo Word | 3.40 | 5.28 | 3.28 | 3.31 | 5.15 |
GPT4 and Sonnet 4.5 exhibited "hyper-duplex" behavior: too much repetition with little thematic reasoning. GPT4o-mini had closer phrase echo strength to Brown, but repeated reframing lines excessively. OLMo 3 differed from Brown in almost all measures.
When provided with Brown's duplex parameters, the models generated poems with higher phrase echo strength, echo contrast, word-level echo strength, and first-last echo, but their excessive repetition contributed little to the poem’s meaning. The “specific” prompt led to the strongest echo behavior; the “figurative” prompt increased the reframing repetition score; and the “general” prompt slightly increased both.
We calculated baseline similarity by conducting pairwise similarity analysis (using cosine similarity) across each set:
Table 4: Average Cosine Similarity Within Poem Sets
| Source | Average Similarity |
| Original Duplexes | 0.39 |
| GPT-4 | 0.40 |
| GPT-4o Mini | 0.45 |
| OLMo 3 | 0.47 |
| Claude 4.5 | 0.38 |
We used the CMU Pronouncing Dictionary to count syllables per line, tracking adherence to the 9-11 syllable constraint by calculating the percentage of lines within this range. GPT-4 achieved 62%, GPT-4o Mini 58%, OLMo 3 42%, and Claude 4.5 64%, compared to Brown's 89%, indicating all models struggled with this formal constraint.
Figure 2. Percent of lines within the 9-11 syllable range per corpus
This project reinforces the distinction between imitation and creation. While genAI models can attempt to follow rules, they struggle to inhabit the emotional, cultural, and historical stakes that give Brown's duplexes their innovation and power. By exaggerating repetition, flattening emotional specificity, and misapplying reframing, the models make visible what human poets like Brown do with intention and skill. The failures of the models illuminate the duplex's use of poetic restraint and lived experience—qualities that cannot be reduced to rules alone. AI-generated poetry functions less as replacement for human creativity and more as diagnostic lens through which poetic form and authorship can be better understood. The gap between Brown's meaning-making power and AI's patterned output underscores both the limits of generative models and the enduring value of human poetic voice.
More generally, this classroom-based approach to computational humanities research, in which undergraduate students produced all of the analysis, both literary and technical, provides a more substantive model for developing literacy about AI and its limits. By engaging undergraduates as active researchers, and by doing so through an examination of a contemporary Black poet, this project expands the range of voices and perspectives in discussions of the value of poetry, and of the value–and limits–of generative AI alike.