Daejeon, July 27–31
Contemporary discussions of gender increasingly revolve around questions of engagement and inclusion: how people relate to categories such as man and woman, how they make room for nonbinary identifications, and how far movement across these positions can be acknowledged and sustained in social life (Butler 1990; Butler 2024; Ahmed 2017; D’Ignazio / Klein 2020). This paper examines how gender-associated traits are also cued and reinforced in language, where particular lexical choices and recurrent contextual patterns invite familiar, often habitual, interpretations of who is being presented as a man, a woman, or as more ambiguously gendered (Butler 1990; Eckert 2000). Over time, these language practices leave cumulative linguistic traces across texts, informing the expectations that cluster around terms such as man, woman, and person, and shaping the range of contexts in which those terms are likely to be read as fixed, as mutable, or as open to alternative construals (D’Ignazio / Klein 2020).
Masked language models such as RoBERTa make it possible to register both the stabilisation of linguistic expectations and the moments at which those expectations are unsettled. Trained to predict words from masked text, these models internalise recurrent distributional regularities across large bodies of text and, in doing so, become sensitive to how socially salient categories are conventionally articulated in language, including terms such as man, woman, and person.
A growing body of work has documented how language models reproduce gender bias, particularly through recurring associations between gendered terms and stereotypical roles or attributes (Bolukbasi et al. 2016; Caliskan et al. 2017; Blodgett et al. 2020; Nangia et al. 2020; Nadeem et al. 2021; Kurita et al. 2019). Much of this research, however, has treated gender categories themselves as stable reference points, focusing on how bias accumulates around them rather than on whether the categories remain internally coherent in language use. As a result, shifts in the semantic scope of the entities to which bias and stereotypes attach have received comparatively limited scrutiny. This gap opens a set of related questions about categorical fluidity: whether man can be positioned, in context, as woman or person and vice versa; whether person consistently resists gendered alignment; how often gender categories preserve, relax, or invert their expected boundaries at the level of individual sentences; and how far these patterns of permeability are genre-dependent, with science fiction, compared with general fiction, potentially producing linguistic environments in which the boundaries stabilising gendered reference are loosened or reconfigured.
The study draws on two literary corpora. The science-fiction dataset contains 336 works from the Gollancz SF Masterworks series (over 90% dated 1910–2000). The comparison dataset is drawn from HathiTrust NovelTM (Underwood et al. 2020) and comprises a randomly sampled set of 700 Anglophone novels published in the same period.
The experimental pipeline extends the Mask & Classify workflow introduced by Liu et al. (2025), which developed the masked-prediction procedure of the Living with Machines project (Ardanuy et al. 2020; Wilson et al. 2023), to the analysis of gendered reference. In each corpus, all sentences containing the target lemmas man/men To distinguish specific-person from generic-mankind uses, man/men sentences were sense-annotated using 3 LLMs and validated against human annotation; only specific-person instances were retained.
Following Liu et al. (2025), each candidate completion is then assigned a semantic label via a Gemini-based classification step. The prompt enforces a gender-focused taxonomy, mapping predictions to four core categories (Male, Female, Neutral-Gender, Nonbinary-Gender), with residual labels (Other-Category, Unknown) for indeterminate cases. The analysis reports retention rates (predictions remaining within the target’s category) and replacement rates (cross-category substitutions). In addition, Shannon entropy is computed over the normalised probability distribution of the top-five predictions for each masked position, providing a measure of how concentrated or diffuse the model’s expectations are for a given target term: lower entropy indicates a narrow, strongly favoured set of candidates, whereas higher entropy signals a broader and more uncertain field of alternatives. Across-corpus differences in retention and replacement are tested using permutation tests with bootstrap confidence intervals; entropy is tested using a two-way ANOVA followed by Bonferroni-corrected Welch post-hoc tests.
Figure 1. Cross-Category Replacement Patterns in Gollancz SF and NovelTM Corpora.
Figure 1 shows a clear ordering in retention rates, exemplifying how strongly each target term is stabilised by its most characteristic category. In NovelTM, person/people exhibits the strongest within-category retention, mapping most often to Neutral-Gender (52.4%), followed by man mapping to Male (41.7%), while woman shows the lowest self-retention, mapping to Female at 34.3%. Gollancz preserves the same ordering, with retention rates of 57.4%, 39.8%, and 34.4% for Neutral-Gender, Male, and Female, respectively.
Of particular interest is the effect of corpus type on the retention rates for the three target terms: person/people becomes more consistently Neutral-Gender in Gollancz than in NovelTM (57.4% in Gollancz versus 52.4% in NovelTM; Δ = +5.0 pp, 95% CI [+3.30, +6.90], p < .001), man shows a small decrease in Male retention in Gollancz relative to NovelTM (39.8% versus 41.7%; Δ = -1.9 pp, 95% CI [-3.78, -0.06], p < .05), and woman remains essentially unchanged across the two corpora (34.4% versus 34.3%; Δ = +0.13 pp, 95% CI [-1.60, +1.89], p = .894, n.s.).
It seems that in the science-fiction corpus, the linguistic environments that host man appear to license a slightly wider set of gendered and non-gendered continuations that compete with the Male-aligned completion, lowering Male retention by about 1.9 percentage points (41.7% to 39.8%). By contrast, the environments that host person/people more reliably support a neutral construal that the model preserves, raising Neutral-Gender retention by about 5 percentage points (52.4% to 57.4%). This stronger neutrality is accompanied by weaker drift into gendered categories in Gollancz (Male 11.4% and Female 5.1%) than in NovelTM (Male 14.9% and Female 7.5%; Male crossover: Δ = -3.5 pp, 95% CI [-4.5, -2.5], p < .001; Female crossover: Δ = -2.4 pp, 95% CI [-3.2, -1.6], p < .001). Across these corpora, science-fiction contexts therefore appear to loosen the contextual anchoring of an explicitly gendered term (man), while also keeping generic reference (person/people) more securely neutral by placing it in contexts that supply fewer gender-skewing cues, which in turn limits the drift of neutral reference toward a masculine default.
A further asymmetry clarifies how categorical permeability operates inside this gradient: bidirectional gender crossover is systematic and sharply uneven. For man, Male retention is about five times the Female crossover in NovelTM (41.7% vs. 8.2%; ≈5:1), and remains above four times in Gollancz (39.8% vs. 9.5%; ≈4:1). For woman, Female retention is roughly twice the Male crossover in both corpora (NovelTM: 34.3% vs. 16.8%; ≈2:1; Gollancz: 34.4% vs. 17.2%; ≈2:1). The same two-directional crossover is available for both targets, yet the imbalance holds across genres: crossover away from Female toward Male is markedly more frequent than crossover away from Male toward Female (Gollancz: Δ = -7.7 pp, 95% CI [-8.6, -6.7], p < .001, Cohen’s d = 0.43; NovelTM: Δ ≈ -8.5 pp, 95% CI [-9.5, -7.5], p < .001, Cohen’s d = 0.49). This asymmetry surfaces the biases latent in the data on which RoBERTa was trained: male is often the default subject of human action and the dominant object of narrative attention, while female is habitually defined in relation to that male norm rather than independently (de Beauvoir 1949; Eckert / McConnell-Ginet 2003). As a result, at the level of distributional prediction, woman proves more susceptible to Male-aligned completions than man is to Female-aligned ones: when the residual co-text after masking does not supply cues strong enough to identify the referent as Female, the model resolves toward Male, consistent with the long-established position of the male as the default subject of action and object of interest. A further source of this crossover lies in authorial choices, particularly in science fiction, where woman is placed in roles whose co-text carries male-coded associations; in such cases, the wrong prediction identifies for us a site of interest for close reading, to see how precisely the author might be challenging convention.
Figure 2. Entropy of Predictions across Target Terms in Gollancz SF and NovelTM Corpora.
Entropy adds a complementary view by tracking how concentrated or diffuse the model’s expectations are at the masked position (Figure 2). Entropy shows that the three target terms are differently ordered across the two corpora in terms of predictability, a pattern supported by a significant category × corpus interaction (F(2, 15,063) = 8.46, p < .001, η² = .0011). In NovelTM, man occupies the lowest-entropy position (H = 1.5832) and is significantly lower than both person/people (H = 1.6504; t = -4.11, p < .001) and woman (H = 1.6563; t = -4.42, p < .001), indicating that masculine reference forms the most securely anticipated position in the general-fiction corpus. Gollancz reorganises this pattern. The NovelTM hierarchy centred on man loses its dominance, while person/people occupies the numerical minimum (H = 1.5931) and shows significantly lower entropy in Gollancz than in NovelTM (H = 1.5931 vs H = 1.6504; Δ = -.057; t = -3.73, p < .001). This lower entropy coincides with the higher Neutral-Gender retention rate for person/people (57.4% in Gollancz vs 52.4% in NovelTM; Δ = +5.0 pp, p < .001), so neutral reference in the science-fiction corpus becomes more stable in both category and prediction: the model is more likely to keep it neutral, and more confident about the lexical possibilities available at the masked position. The entropy changes for man and woman remain descriptive rather than statistically significant after correction for multiple comparisons: man is numerically higher in Gollancz than in NovelTM (H = 1.6181 vs H = 1.5832; Δ = +.035; t = 2.04, raw p = .041), while woman is numerically lower (H = 1.6246 vs H = 1.6563; Δ = -.032; t = -2.01, raw p = .045), with neither comparison surviving Bonferroni correction. The replacement results (Figure 1) show the complementary side of this reorganisation: in Gollancz, the explicitly gendered targets also move more often toward Neutral-Gender than in NovelTM, with higher Neutral-Gender crossover for man (30.1% versus 27.7%; Δ = +2.4 pp, 95% CI [+0.7, +4.2], p < .01, Cohen’s d = 0.08) and woman (26.8% versus 24.2%; Δ = +2.6 pp, 95% CI [+1.2, +4.1], p < .001, Cohen’s d = 0.10). The larger pattern is therefore one of genre-level neutralisation. General fiction organises predictability most strongly around man; science fiction weakens that masculine centre and shifts the clearest point of stabilisation toward neutral person-reference.
These findings show that genre modulates the semantic scope of gendered reference in target-specific ways: science-fiction contexts weaken Male anchoring for man, strengthen the neutrality of person/people, and leave woman broadly stable, while nonbinary reference appears only sporadically in model completions. What counts as gender "inclusion" or "movement" in social life does not translate automatically into linguistic availability: whether a given way of being gendered can find expression in a given context is decided by the accumulated distributional force of prior usage. If the language we read and write shapes the categories we find it customary to think in, as a substantial body of cognitive research suggests (Boroditsky 2011), then the distributional patterns documented here carry consequences that reach beyond the text: a language that leaves little room for certain gendered possibilities at the level of context may, over time, make those possibilities harder to imagine, while the stereotypes and biases already encoded in its patterns continue to deepen their hold on how gender is perceived and lived.
The broader methodological contribution of this study lies in its development of Mask & Classify as a portable research tool for literary and digital humanities scholarship. Language models are now powerful instruments for literary studies because they can make large-scale patterns of meaning visible without displacing human interpretation. When specific words are masked, the model’s predicted substitutes reveal the semantic expectations embedded in literary language; when those substitutes are then classified, these expectations can be measured as patterns of retention, crossover, dispersion, and categorical instability. Mask & Classify therefore treats masked language modelling not as text generation, but as an interpretive procedure for examining phenomena that are often discussed qualitatively. Liu et al. (2025), for example, showed that when human, animal, and machine are masked, the resulting predictions can be used to analyse estrangement, anthropomorphism, posthuman ambiguity, and the instability of ontological categories. The present study extends that logic to gendered reference, showing how man, woman, and person are differentially stabilised or opened to crossover in fiction. In this sense, Mask & Classify helps scholars move between close and distant reading: it identifies statistically unusual or “wrong” predictions that point critics toward moments of textual surprise, conceptual disruption, and genre-specific meaning. Its power lies not in reading literature for us, but in exposing patterns of deviation, hierarchy, and semantic tension that human readers can then interpret critically.
We gratefully acknowledge support from Riksbankens Jubileumsfond, Change is Key! (Grant No. M21-0021; award to Haim Dubossarsky), and from the QMUL–CSC PhD Scholarship program (China Scholarship Council; Grant No. 202408890010; award to Yuxuan Liu).