DH 2026

Daejeon, July 27–31

Thu, July 3016:30–18:00S069107
Short Paper

Beyond Similarity: Bridging the Interpretability Gap in Intertextual Analysis with LLM-Driven Annotation — A Case Study of the Analects and the Book of Han

Zhaoji Wang
Department of Information Management, Peking University, Beijing, 100871, China; Research Center for Digital Humanities, Peking University, Beijing, 100871, China · zhaojiwang@stu.pku.edu.cn
Wanyu Si
Department of Information Management, Peking University, Beijing, 100871, China; Research Center for Digital Humanities, Peking University, Beijing, 100871, China · erin@stu.pku.edu.cn
Jun Wang
Department of Information Management, Peking University, Beijing, 100871, China; Research Center for Digital Humanities, Peking University, Beijing, 100871, China · junwang@pku.edu.cn

Introduction

Intertextuality is fundamental to understanding intellectual transmission and canon formation (Kristeva 1980). While Digital Humanities has made significant strides in text reuse detection—using algorithms from n-grams to vector embeddings to identify parallel passages across massive corpora (Sturgeon 2018; Duan et al. 2023; Wang et al. 2024)—a critical interpretability gap persists. Existing computational approaches detect that texts are related but rarely explain how or why (Forstall / Scheirer 2019). A high similarity score between the Analects (Lunyu) and a historical chronicle does not distinguish whether a citation serves as a theoretical endorsement, a practical policy application, or a target of critique. As Periti et al. (2024) noted, models remain sensitive to textual overlap rather than topic relatedness without structured guidance.

This study addresses this gap by proposing a multi-dimensional annotation framework that moves from detection to interpretation. Responding to the DH2026 sub-theme “Annotating | Beyond Patterns,” we demonstrate how a Large Language Model (LLM) agent, rigorously verified by domain experts within a human-in-the-loop workflow, can transform large-scale text data into a high-quality “Small Data” set. This allows us to decode the rhetorical functions and political stances embedded in classical intertextuality—dimensions previously invisible to frequency-based analysis.

Related Work

Quantitative intertextuality has evolved from lexical matching (Coffee et al. 2012) to semantic capture via neural approaches (Manjavacas et al. 2019; Duan 2025). In Chinese studies, projects like Evol (Wang et al. 2024) and comparative analyses of the Shiji and Hanshu (Deng et al. 2022) have successfully mapped macroscopic cultural evolution via entity recognition. However, these contributions largely remain within the similarity paradigm, mapping who is mentioned without discerning the stance or purpose.

Building on citation function analysis (Jurgens et al. 2018) and recent schemas for philosophical texts (Yang et al. 2025), our work extends functional annotation to classical Chinese historiography within a human-in-the-loop paradigm. Recent studies have validated LLMs as effective annotation tools when paired with structured guidelines and human oversight (Rathje et al. 2024). Unlike modern academic citations, classical intertextuality involves unstandardized practices and subtle rhetorical encoding, posing unique challenges that motivate our LLM-agent-based approach.

Method: A Human–LLM Collaborative Framework

We conducted an exhaustive pairwise comparison between the Analects (20 chapters) and the Book of Han (Hanshu, approximately 120 textual units across 100 scrolls), generating approximately 2,400 comparison tasks. Each task was processed by an LLM agent powered by Doubao-Seed-1.6 (Bytedance), a sparse Mixture-of-Experts model selected for its strong classical Chinese comprehension and 256K-token context window. The agent received both full texts together with a structured analytical protocol comprising exclusion criteria, definitions and decision rules for all seven dimensions (annotated jointly in a single prompt per task), and a text-positioning tool that forced every annotation to be grounded in verifiable textual evidence. Automated checks then validated the JSON structure, exact text matches, character offsets, and field completeness before expert review. Each task could yield zero, one, or multiple intertextual pairs; the final dataset comprises 464 validated pairs.

Figure 1. Seven-dimensional annotation framework combining surface features (Form, Aspect, Granularity) with interpretive analysis (Signaling, Function, Stance, Evidence Strength) in a Human–LLM workflow.

Function captures rhetorical role; Signaling records attribution style; Evidence Strength distinguishes clear textual dependence from probable dependence or thematic echo.

Findings: The Politics of Citation

Finding 1: From theory to practice. While similarity-based methods count occurrences, our functional analysis reveals usage. We found that 37.3% of citations serve the Application function, transforming abstract ethics into concrete administrative protocols (e.g., tax exemptions for the pious). For instance, the Analects maxim “To execute without first educating is cruelty” reappears in Dong Zhongshu’s biography as an explicit policy critique of the court’s over-reliance on punitive law—a functional transformation invisible to similarity metrics. Moreover, the most cited chapter is not the theoretical Xue Er but Wei Ling Gong (39 times), rich in governance strategies, confirming a pragmatic turn in the Book of Han’s engagement with the canon.

Finding 2: Implicit hegemony via unmarked signaling. Our data reveals that 60.1% of citations are unmarked (used without attribution), while only 31.7% are explicitly named. This distribution suggests that by the time of Hanshu’s compilation, Analects discourse had become deeply internalized into the intellectual vocabulary. Unmarked citations indicate these concepts functioned as shared “common sense” requiring no explicit acknowledgment. Paradoxically, the lack of explicit citation signals a higher form of authority: an ideological hegemony where the source text becomes pervasive and invisible.

Finding 3: Genre and “legitimation through practice.” By calculating Standardized Citation Density (citations per 10,000 characters) to control for the substantial variation in text length across genres, we discovered a significant genre bias (see Figure 2).

Figure 2. Citation density by Book of Han genre, normalized as citations per 10,000 characters (N=464), with proportional deviation from text share.

Biographies exhibit the highest character-normalized density and the only positive proportional deviation, meaning their share of citations (75.6%) substantially exceeds their share of text (66.3%). This supports a theory of Legitimation through Practice: The Han dynasty established Confucian orthodoxy not primarily through top-down imperial decrees, but through the lived examples of officials and scholars in Biographies. The legitimacy of the canon was thus validated through human administrative practice rather than abstract proclamation.

Discussion and Conclusion

This study demonstrates that LLMs, guided by rigorous schemas and human expertise, can bridge the gap between “distant” and “close” reading. By annotating Function and Signaling, we move beyond detecting physical reuse to understanding the social life of ideas.

Our approach reveals that the canonization of the Analects was a complex operation of functional transformation (philosophy to policy), rhetorical internalization (explicit to implicit), and generic distribution (validating ideology through biographical narratives). This framework offers a reproducible schema for “Computational Hermeneutics,” inviting the community to reconsider the value of annotated “Small Data” in the age of LLMs.

References
  1. Burns, Patrick J. / Brofos, James A. / Li, Kyle / Chaudhuri, Pramit / Dexter, Joseph P. (2021): “Profiling of intertextuality in Latin literature using word embeddings”, in: Proceedings of NAACL-HLT 2021: 4900–4907.
  2. Coffee, Neil / Koenig, Jean-Pierre / Poornima, Shakthi / Ossewaarde, Roelant / Forstall, Christopher / Jacobson, Sarah (2012): “Intertextuality in the digital age”, in: Transactions of the American Philological Association 142, 2: 383–422.
  3. Deng, Zekun / Yang, Hao / Wang, Jun (2022): “A comparative study of Shiji and Hanshu from the perspective of digital humanities”, in: Proceedings of CCL 2022: 656–670.
  4. Duan, Siyu (2025): “Quantitative intertextuality from the digital humanities perspective: a survey”, arXiv:2510.27045.
  5. Duan, Siyu / Wang, Jun / Yang, Hao / Su, Qi (2023): “Disentangling the cultural evolution of ancient China: a digital humanities perspective”, in: Humanities and Social Sciences Communications 10: 310.
  6. Forstall, Christopher W. / Scheirer, Walter J. (2019): Quantitative Intertextuality. Springer.
  7. Jurgens, David / Kumar, Srijan / Hoover, Raine / McFarland, Dan / Jurafsky, Dan (2018): “Measuring the evolution of a scientific field through citation frames”, in: Transactions of the Association for Computational Linguistics 6: 391–406.
  8. Kristeva, Julia (1980): “Word, dialogue, and novel”, in: Roudiez, Leon S. (ed.): Desire in Language: A Semiotic Approach to Literature and Art. New York: Columbia University Press: 64–91.
  9. Manjavacas, Enrique / Long, Brian / Kestemont, Mike (2019): “On the feasibility of automated detection of allusive text reuse”, in: Proceedings of LaTeCH-CLfL 2019: 104–114.
  10. Periti, Francesco / Cassotti, Pierluigi / Montanelli, Stefano / Tahmasebi, Nina / Schlechtweg, Dominik (2024): “TRoTR: A framework for evaluating the re-contextualization of text reuse”, in: Proceedings of EMNLP 2024: 13972–13990.
  11. Rathje, Steve / Mirea, Dan-Mircea / Sucholutsky, Ilia / Marjieh, Raja / Robertson, Claire E. / Van Bavel, Jay J. (2024): “GPT is an effective tool for multilingual psychological text analysis”, in: Proceedings of the National Academy of Sciences 121, 34: e2308950121.
  12. Sturgeon, Donald (2018): “Unsupervised identification of text reuse in early Chinese literature”, in: Digital Scholarship in the Humanities 33, 3: 670–684.
  13. Wang, Jun / Duan, Siyu / Fu, Binghao / Gao, Liangcai / Su, Qi (2024): “Evol project: a comprehensive online platform for quantitative analysis of ancient literature”, in: Humanities and Social Sciences Communications 11: 291.
  14. Yang, Yue / Xu, Yinzhi / Huang, Chenghao / Jurgensen, JohnMichael / Hu, Han / Wang, Hao (2025): “InterIDEAS: Philosophical intertextuality via LLMs”, in: Proceedings of EMNLP 2025: 23151–23172.