DH 2026

Daejeon, July 27–31

Thu, July 3013:40–15:10S064106
Short Paper

What LLMs Know About Humanistic Theory: Challenges of Theoretical Plurality in the Age of AI

Rabea Kleymann
TU Chemnitz, Germany · rabea.kleymann@phil.tu-chemnitz.de

Introduction

Humanistic theories serve as essential frameworks for interpreting phenomena, shaping how scholars understand research fields and formulate questions (Suppe 1998; Abend 2008; Zima 2007; Corvellec 2013). As Johanna Drucker (Suppe 1998; Abend 2008; Zima 2007; Corvellec 2013). As Johanna Drucker (2012) states: “Humanistic theory provides ways of thinking differently, otherwise, specific to the problems and precepts of interpretative knowing – partial, situated, enunciative, subjective, and performative.” Moreover, it is the plurality of theories, which is often regarded as a characteristic feature of the humanities (2012) states: “Humanistic theory provides ways of thinking differently, otherwise, specific to the problems and precepts of interpretative knowing – partial, situated, enunciative, subjective, and performative.” Moreover, it is the plurality of theories, which is often regarded as a characteristic feature of the humanities (Alvarado 2012; Kleymann et al. 2022; Kleymann 2024). As large language models (LLMs) increasingly enter research workflows within digital humanities (DH) scholarship, they influence or even distort practices of theorizing (Alvarado 2012; Kleymann et al. 2022; Kleymann 2024). As large language models (LLMs) increasingly enter research workflows within digital humanities (DH) scholarship, they influence or even distort practices of theorizing (Kirschenbaum 2023; Karjus 2023). At the same time, LLMs are presented as theory-free (Kirschenbaum 2023; Karjus 2023). At the same time, LLMs are presented as theory-free (Spinney 2022; Andrews 2025). While LLMs’ impact on interpretation in DH has been studied, the role of humanistic theories remains underexamined (Spinney 2022; Andrews 2025). While LLMs’ impact on interpretation in DH has been studied, the role of humanistic theories remains underexamined (Jannidis et al. 2025): What kind of theories do LLMs replicate potentially impacting the plurality of humanistic theories? And how could we study what LLMs know about humanistic theory? (Jannidis et al. 2025): What kind of theories do LLMs replicate potentially impacting the plurality of humanistic theories? And how could we study what LLMs know about humanistic theory?

This short paper seeks to investigate the impact of LLMs on humanistic theorizing. I argue that LLMs are highly theory-laden. They are modeled and fine-tuned by theoretically imbued data. This means that the data carries inherent theoretical assumptions and perspectives that shape how the LLM understands and generates theories. This “theory-ladenness” (Hanson 1965; Kuhn 1994; Leonelli 2020) of LLMs as cultural and sociotechnical artifacts needs to be reflected within the DH community. As work in progress, this short paper contributes to discussions on how DH scholars might critically engage with and against LLMs through humanistic theory (Hanson 1965; Kuhn 1994; Leonelli 2020) of LLMs as cultural and sociotechnical artifacts needs to be reflected within the DH community. As work in progress, this short paper contributes to discussions on how DH scholars might critically engage with and against LLMs through humanistic theory (Raley and Rhee 2023; Lindgren 2024). It invites DH scholars to reflect on shared practices of theorizing within our research community. Methodologically, I proceed as follows: starting with the semantic ambiguity of (Raley and Rhee 2023; Lindgren 2024). It invites DH scholars to reflect on shared practices of theorizing within our research community. Methodologically, I proceed as follows: starting with the semantic ambiguity of theory and theorizing in science and humanities, I describe four challenges for humanistic theorizing in the age of LLMs. In a second step, I shortly present one probing case study. It tries to reconstruct theory as a “machinic construct” (Offert and Phan 2022) through a reflective prompting approach. (Offert and Phan 2022) through a reflective prompting approach.

Humanistic Theory and its Plurality

The term theory has its etymological roots in the Greek verb “theorein (θεωρεῖν),” which translates to “to observe,” “to contemplate,” or “to see” (Liddell and Scott 1889). Although (Liddell and Scott 1889). Although theory is a foundational concept across scholarly domains, it remains polysemous. Neither the sciences nor the humanities have reached consensus on a single definition of theory, nor on shared criteria for how theories ought to be constructed or validated (Kuhn 1994; Nuzzo 1999). Theories serve as fundamental components of research, mediating the relationship between an object of study and research communities (Kuhn 1994; Nuzzo 1999). Theories serve as fundamental components of research, mediating the relationship between an object of study and research communities (Martus and Spoerhase 2022). What is so special about humanistic theories is on the one hand that they are not formalized, for example in an hypothesis or model. Rather they exhibit a “semantic-narrative” (Martus and Spoerhase 2022). What is so special about humanistic theories is on the one hand that they are not formalized, for example in an hypothesis or model. Rather they exhibit a “semantic-narrative” (Zima 2007) or discursive structure (Zima 2007) or discursive structure (Winther 2021). On the other hand, humanistic theories can be regarded as “written articulations of complex ideas” (Winther 2021). On the other hand, humanistic theories can be regarded as “written articulations of complex ideas” (Klein et al. 2025: 3). They often emphasize context-sensitive accounts of phenomena. For the purpose of this paper, humanistic theory is understood as a set of discursively structured, non-formalized, context-sensitive interpretive frameworks that mediate between research objects and disciplinary communities in the humanities.(Klein et al. 2025: 3). They often emphasize context-sensitive accounts of phenomena. For the purpose of this paper, humanistic theory is understood as a set of discursively structured, non-formalized, context-sensitive interpretive frameworks that mediate between research objects and disciplinary communities in the humanities.

Four Epistemic Challenges

Against this backdrop, LLMs introduce four epistemic challenges. First, although LLMs are often framed as theory-free, their training data embed theoretical commitments. Second, LLMs do not store knowledge as formal, rule-based symbolic representations of the world. Instead, they function as statistical, subsymbolic models where knowledge is implicitly encoded in training datasets, architectural components, and fine-tuning (Li et al. 2021; Holton 2023; Sudmann et al. 2023). As a consequence, LLMs seem to shift the relation between theory and data by replacing interpretive reasoning with statistical prediction. Third, their outputs may homogenize theoretical approaches by amplifying canonical, predominantly Western and Anglophone theories while marginalizing traditions from the so-called Global South (Li et al. 2021; Holton 2023; Sudmann et al. 2023). As a consequence, LLMs seem to shift the relation between theory and data by replacing interpretive reasoning with statistical prediction. Third, their outputs may homogenize theoretical approaches by amplifying canonical, predominantly Western and Anglophone theories while marginalizing traditions from the so-called Global South (Risam and Josephs 2021; Fiormonte et al. 2022) . This “epistemic bias” (Risam and Josephs 2021; Fiormonte et al. 2022) . This “epistemic bias” (Bueter 2022; Zermeño-Flores et al. 2024) might reduce the plurality of theories within AI-assisted workflows. Finally, because training data and fine-tuning remain inaccessible, DH scholars cannot easily assess which theoretical frameworks are encoded or excluded (except of (Bueter 2022; Zermeño-Flores et al. 2024) might reduce the plurality of theories within AI-assisted workflows. Finally, because training data and fine-tuning remain inaccessible, DH scholars cannot easily assess which theoretical frameworks are encoded or excluded (except of WIMBD by Elazar et al. 2023; Elazar et al. 2023; Patchscopes by Ghandeharioun et al. 2024). Together, these challenges have an impact on how to use LLMs in DH research.Ghandeharioun et al. 2024). Together, these challenges have an impact on how to use LLMs in DH research.

Probing Case Study

The case study investigates how three LLMs represent humanistic theory and theorizing. It examines which theoretical representations the models inherently possess and how these are augmented through a reflective probing framework. The models used are GPT5.1 (OpenAI), Gemini 3 (Google) and Mistral Large 3.

The models were selected to enable a comparative analysis across different institutional and infrastructural contexts. For the presentation, more up-to-date models may be used, which then comply with current standards.

In this context probing is understood as a systematic prompting method to elicit and compare a LLM’s latent theoretical assumptions. In relation to benchmarking, this probing design aligns with Kommers et al.’s (2025) call for critical evaluation rather than accuracy-based scoring. Prompts are generated for the tasks of defining, application and comparing humanistic theories. Initial results show the near-identical top-ten ranking of theories (“Structuralism/Post-structuralism, Marxism (Critical Theory), Psychoanalysis, Feminist Theory, Postcolonialism, Hermeneutics, Phenomenology, Deconstruction, New Historicism/Cultural Materialism, Queer Theory”).(2025) call for critical evaluation rather than accuracy-based scoring. Prompts are generated for the tasks of defining, application and comparing humanistic theories. Initial results show the near-identical top-ten ranking of theories (“Structuralism/Post-structuralism, Marxism (Critical Theory), Psychoanalysis, Feminist Theory, Postcolonialism, Hermeneutics, Phenomenology, Deconstruction, New Historicism/Cultural Materialism, Queer Theory”).

Prompt example: “From the perspective of humanities and digital humanities scholarship, list and rank the ten most influential humanistic theories. Please provide a ranked list (1–10) based on their prominence, impact, and continued relevance in scholarly discourse. Do not tailor the ranking to a specific discipline or national tradition; aim for a broadly recognized humanities perspective. Provide only the ranked list, without additional explanation.” This prompt was applied unchanged across all three models and prompting conditions to ensure comparability (e.g. temperature).

The ranking across models suggests that LLMs encode a shared representation of theories. Notably, these theories are often represented in coursebooks and introductions to literary and critical theories (Tyson 2006; Rivkin and Ryan 2007; Dobie 2012). This convergence, however, could support the thesis that LLMs are not theory-neutral but reproduce historically dominant and institutionally sedimented theoretical frameworks. Furthermore, multi-prompt probing is conducted across zero-shot, few-shot, and stability tests to observe whether these three models maintain coherent theoretical representations when asked to interpret texts through different humanistic lenses.(Tyson 2006; Rivkin and Ryan 2007; Dobie 2012). This convergence, however, could support the thesis that LLMs are not theory-neutral but reproduce historically dominant and institutionally sedimented theoretical frameworks. Furthermore, multi-prompt probing is conducted across zero-shot, few-shot, and stability tests to observe whether these three models maintain coherent theoretical representations when asked to interpret texts through different humanistic lenses.

Prompt example: “Given the following literary text: Johann Wolfgang von Goethe, Die Wahlverwandtschaften (1819). Task 1: Provide an interpretation (150–200 words) of the text from each of the following literary-theoretical perspectives: (a) Marxism, (b) Poststructuralism, (c) Queer theory. For each interpretation, explicitly articulate the key theoretical assumptions guiding the analysis and indicate which textual features are foregrounded. Task 2: After generating the three interpretations, rank them according to how plausibly they represent their respective theoretical frameworks. Discuss which theoretical perspective is most convincingly emulated by the model and justify the ranking with reference to core theoretical concepts and their application to the text” (Santurkar et al., 2023; Ziems et al., 2024).(Santurkar et al., 2023; Ziems et al., 2024).

The models seem to diverge in how theoretical concepts are operationalized in textual interpretation, indicating that theory functions less as a coherent epistemic framework and more as a flexible rhetorical scaffold with certain keywords (e.g. “différance”). These initial findings do not demonstrate epistemic bias in a normative sense yet but might point to structural tendencies toward theoretical standardization. Further analyses of the case study, based on additional examples, will be shortly presented in the talk.

References
  1. Abend, Gabriel (2008): “The Meaning of Theory”, in: Sociological Theory 26, 2: 173–199. DOI: 10.1111/j.1467-9558.2008.00324.x.
  2. Alvarado, Rafael C. (2012): “The Digital Humanities Situation”, in: Gold, Matthew K. (ed.): Debates in the Digital Humanities. Minneapolis: University of Minnesota Press.
  3. Andrews, Mel (2025): “The Immortal Science of ML: Machine Learning and the Theory-Free Ideal”, in: Erkenntnis. DOI: 10.1007/s10670-025-01010-x.
  4. Bueter, Anke (2022): “Bias as an Epistemic Notion”, in: Studies in History and Philosophy of Science 91: 307–315. DOI: 10.1016/j.shpsa.2021.12.002.
  5. Corvellec, Hervé (ed.) (2013): What Is Theory? Answers from the Social and Cultural Sciences. Copenhagen: Liber / Copenhagen Business School Press.
  6. Dobie, Ann Brewster (2012): Theory into Practice: An Introduction to Literary Criticism. 3rd edition. Boston: Wadsworth Cengage Learning.
  7. Drucker, Johanna (2012): “Humanistic Theory and Digital Scholarship”, in: Gold, Matthew K. (ed.): Debates in the Digital Humanities. Minneapolis: University of Minnesota Press.
  8. Elazar, Yanai / Bhagia, Akshita / Magnusson, Ian et al. (2023): “What’s In My Big Data?”, Preprint, arXiv. DOI: 10.48550/ARXIV.2310.20707.
  9. Ghandeharioun, Asma / Caciularu, Avi / Pearce, Adam / Dixon, Lucas / Geva, Mor (2024): “Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models”, Preprint, arXiv. DOI: 10.48550/ARXIV.2401.06102.
  10. Hanson, Norwood Russell (1965): Patterns of Discovery: An Inquiry into the Conceptual Foundations of Science. Cambridge: Cambridge University Press.
  11. Holton, Robert (2023): “Artificial Intelligence and the Problem of Radical Uncertainty”, in: Lindgren, Simon (ed.): Handbook of Critical Studies of Artificial Intelligence. Cheltenham: Edward Elgar Publishing.
  12. Hussein, Nada / Ghandeharioun, Asma / Mullins, Ryan et al. (2024): “Can Large Language Models Explain Their Internal Mechanisms?”, in: People + AI Research (PAIR), July 2024 https://pair.withgoogle.com/explorables/patchscopes/ [07.05.2026].
  13. Jannidis, Fotis / Kleymann, Rabea / Schröter, Julian / Zinsmeister, Heike (2025): “Do Large Language Models Understand Literature? Case Studies and Probing Experiments on German Poetry”, in: Journal of Computational Literary Studies 4, 1. DOI: 10.48694/JCLS.4225.
  14. Karjus, Andres (2023): “Machine-Assisted Mixed Methods: Augmenting Humanities and Social Sciences with Artificial Intelligence”, Preprint, arXiv:2309.14379. DOI: 10.48550/arXiv.2309.14379.
  15. Kirschenbaum, Matthew (2023): “Again Theory: A Forum on Language, Meaning, and Intent in the Time of Stochastic Parrots”, in: Critical Inquiry – In the Moment, 26 June 2023 https://critinq.wordpress.com/2023/06/ [07.05.2026].
  16. Klein, Lauren / Martin, Meredith / Brock, André et al. (2025): “Provocations from the Humanities for Generative AI Research”, Preprint, arXiv. DOI: 10.48550/ARXIV.2502.19190.
  17. Kleymann, Rabea (2024): “Theorie”, in: Begriffe der Digital Humanities. Ein diskursives Glossar. Zeitschrift für digitale Geisteswissenschaften / Working Papers 2. DOI: 10.17175/wp_2023_013_v2.
  18. Kleymann, Rabea / Burghardt, Manuel / Geiger, Jonathan D. / Schumacher, Mareike (2022): “Foreword to the Special Issue 'Theorytellings: Epistemic Narratives in the Digital Humanities'”, in: Journal of Cultural Analytics 7, 4. DOI: 10.22148/001c.55593.
  19. Kommers, Cody / Ahnert, Ruth / Antoniak, Maria et al. (2025): “Computational Hermeneutics: Evaluating Generative AI as a Cultural Technology”, Preprint. DOI: 10.2139/ssrn.5409144.
  20. Kuhn, Thomas S. (1994): The Structure of Scientific Revolutions. Chicago: University of Chicago Press.
  21. Leonelli, Sabina (2020): “Scientific Research and Big Data”, in: Zalta, Edward N. (ed.): The Stanford Encyclopedia of Philosophy. Summer 2020 Edition. Stanford: Stanford University https://plato.stanford.edu/archives/sum2020/entries/science-big-data/ [07.05.2026].
  22. Li, Belinda Z. / Nye, Maxwell / Andreas, Jacob (2021): “Implicit Representations of Meaning in Neural Language Models”, Preprint, arXiv. DOI: 10.48550/ARXIV.2106.00737.
  23. Liddell, Henry George / Scott, Robert (1889): An Intermediate Greek-English Lexicon: Founded upon the Seventh Edition of Liddell and Scott’s Greek-English Lexicon. New York: Harper & Brothers https://www.loc.gov/resource/gdcmassbookdig.intermediategree00lidd/ [07.05.2026].
  24. Lindgren, Simon (2024): Critical Theory of AI. Cambridge: Polity Press.
  25. Martus, Steffen / Spoerhase, Carlos (2022): Geistesarbeit: eine Praxeologie der Geisteswissenschaften. Berlin: Suhrkamp.
  26. Nuzzo, Angelica (1999): “Theorie”, in: Sandkühler, Hans Jörg (ed.): Enzyklopädie Philosophie, vol. 2. Hamburg: Meiner.
  27. Offert, Fabian / Phan, Thao (2022): ”A Sign That Spells: DALL-E 2, Invisual Images and The Racial Politics of Feature Space”, Preprint, arXiv. DOI: 10.48550/ARXIV.2211.06323.
  28. Raley, Rita / Rhee, Jennifer (2023): “Critical AI: A Field in Formation”, in: American Literature 95, 2: 185–204. DOI: 10.1215/00029831-10575021.
  29. Rivkin, Julie / Ryan, Michael (eds.) (2007): Literary Theory: An Anthology. 2nd edition. Malden, MA: Blackwell Publishing.
  30. Santurkar, Shibani / Durmus, Esin / Ladhak, Faisal / Lee, Cinoo / Liang, Percy / Hashimoto, Tatsunori (2023): “Whose Opinions Do Language Models Reflect?”, Preprint, arXiv. DOI: 10.48550/ARXIV.2303.17548.
  31. Spinney, Laura (2022): “Are We Witnessing the Dawn of Post-Theory Science?”, in: The Guardian, 9 January 2022 https://www.theguardian.com/technology/2022/jan/09/are-we-witnessing-the-dawn-of-post-theory-science [07.05.2026].
  32. Sudmann, Andreas / Echterhölter, Anna / Ramsauer, Markus / Retkowski, Fabian / Schröter, Jens / Waibel, Alexander (eds.) (2023): Beyond Quantity: Research with Subsymbolic AI. Bielefeld: Transcript.
  33. Suppe, Frederick (1998): “Theories, Scientific”, in: Craig, Edward (ed.): Routledge Encyclopedia of Philosophy. London: Routledge.
  34. Tyson, Lois (2006): Critical Theory Today: A User-Friendly Guide. 2nd edition. New York: Routledge.
  35. Winther, Rasmus Grønfeldt (2021): “The Structure of Scientific Theories”, in: Zalta, Edward N. (ed.): The Stanford Encyclopedia of Philosophy. Spring 2021 Edition. Stanford: Stanford University https://plato.stanford.edu/archives/spr2021/entries/structure-scientific-theories/ [07.05.2026].
  36. Zermeño-Flores, Ana Isabel / Tufte, Thomas / Navarrete-Vega, Mabel Andrea (2024): “Knowledge Production and Epistemic Injustices: The Use of Digital Technologies and Artificial Intelligence”, in: MedieKultur: Journal of Media and Communication Research 40, 76: 120–138. DOI: 10.7146/mk.v40i76.143529.
  37. Ziems, Caleb / Held, William / Shaikh, Omar / Chen, Jiaao / Zhang, Zhehao / Yang, Diyi (2024): “Can Large Language Models Transform Computational Social Science?”, in: Computational Linguistics: 1–55. DOI: 10.1162/coli_a_00502.
  38. Zima, Peter Václav (2007): What Is Theory: Cultural Theory as Discourse and Dialogue. London: Continuum.