Daejeon, July 27–31
Natural Language Processing (NLP) for historical languages confronts a distinctive cluster of problems: data scarcity, orthographic variation, and fragmentary source materials. Although general-purpose Large Language Models (LLMs) such as GPT-5 and Gemini 3.0 perform strongly on high-resource modern languages, they hallucinate and degrade markedly on low-resource ancient corpora, including the diachronic continuum of Ancient Egyptian–Coptic (Old, Middle, and Late Egyptian, Demotic, and Coptic). Recent work has shown that Retrieval-Augmented Generation (RAG) can mitigate these failures by grounding generation in curated external resources: see Chang et al. (2025) on Hakka and Torres Aguilar (2025) on draft-and-refine RAG for Latin.
This paper introduces THOTH AI, a RAG system that couples Claude Sonnet 4.5 with vectorized lexical and grammatical resources via the Dify platform, and evaluates it on two complementary philological tasks: (i) Neural Machine Translation (NMT) of Middle Egyptian, and (ii) intertextuality detection in Coptic monastic literature. We do not claim that THOTH AI performs interpretation; rather, we ask to what extent retrieval over scholarly resources shifts the locus of LLM error from lexical hallucination toward residual interpretive ambiguity, where philological judgment remains indispensable.
THOTH AI is built on Dify and serves Claude Sonnet 4.5 augmented at inference with a vector store containing approximately 3 million tokens drawn from openly licensed resources: the Comprehensive Coptic Lexicon, Coptic SCRIPTORIUM annotated corpora (Zeldes & Schroeder 2014–25), the Thesaurus Linguae Aegyptiae lemma list, and selected reference grammars and concordances. Content was chunked at lemma- and sentence-level granularity, embedded with OpenAI text-embedding-3-large (3,072 dimensions), and retrieved by cosine similarity (top-k = 8) followed by maximal-marginal-relevance reranking. Indexing the present corpus on the Dify Professional tier required approximately USD 1 in embedding cost (text-embedding-3-large at USD 0.13 per million tokens); per-query inference cost is dominated by the Claude API and averages USD 0.04 per translation request and USD 0.03 per intertextuality query. The end-user web deployment is offered free under a managed quota; preprocessing scripts and the retrieval index will be released as open data upon publication.
Following Miyagawa (2025), we evaluate THOTH AI against Gemini 2.0 (Pro and Flash), DeepSeek-R1, GPT-4o, and the unaugmented Claude 3.5 Sonnet baseline on the coronation decree of the 18th-Dynasty pharaoh Thutmose I (Urk. IV, 79–81). Surface-form metrics—SacreBLEU, METEOR, ROUGE-L, chrF—are reported for comparability with prior work, alongside an Egyptologist’s qualitative scoring of lexical accuracy, syntactic faithfulness, and treatment of formulaic phraseology. THOTH AI achieves SacreBLEU 35.43 against 32.46 for unaugmented Claude 3.5 Sonnet and 19.62 for GPT-4o. We note explicitly that BLEU-family metrics, designed for closed parallel corpora of modern languages, are imperfect proxies for literary translation quality (cf. Torres Aguilar 2025); they are reported here only for comparability and are accompanied by qualitative analysis. Qualitatively, retrieval suppresses lexical hallucination on rare items (royal titulary, ritual vocabulary) and improves the rendering of formulaic phraseology; syntactic disambiguation of long subordinate clauses remains a residual challenge. For the phrase ḏd-mdw in coronation context, the system most frequently retrieved Wb. V, 618.7–9 entries together with Karnak inscriptional parallels indexed in the TLA, allowing a contextually appropriate rendering rather than the generic “words spoken” returned by the unaugmented baseline.
We focus on biblical reuse in On the Punishment of Sinners by Besa, the fifth-century successor of Shenoute as abbot of the White Monastery (Coptic text: Kuhn 1956a = CSCO 157/Copt. 21; English translation: Kuhn 1956b = CSCO 158/Copt. 22). We compare THOTH AI against the manual identifications of Kuhn (1956), the n-gram-based detector TRACER (Büchler et al. 2014–19), and the Smith–Waterman aligner Passim (Smith 2012–23). On a curated test set of 18 candidate biblical intertexts, recall is: THOTH AI 16, Passim 12, TRACER 11, manual scholarship 9. These headline figures should be read with care: intertextuality is not a binary relation but a graded continuum running from verbatim quotation through paraphrase to allusion and conceptual echo (Hays 1989). Our test set is annotated on a four-point scale, and disaggregated results show that THOTH AI’s gain over alignment-based methods is concentrated on the allusion and echo tiers, where lexical overlap is partial or absent.
A representative case is the relation between Besa 3:1:2, “I will trample with my shoe upon Edom,” and Ps 59:10 LXX (= MT 60:8), “I will stretch out my shoe over Edom.” String- and n-gram-based methods miss the link because the Coptic verbs for “trample” and “stretch out” do not overlap. THOTH AI surfaces the parallel and supplies a justification grounded in the shared image of divine judgment against Edom and the structural parallelism with surrounding verses. Whether to accept such a candidate remains a philologist’s decision; the system’s task is to widen the recall of candidates worth evaluating, not to adjudicate them.
THOTH AI is being extended with a Coptic Composition Assistant providing real-time grammatical and stylistic feedback for active language production. As an exploratory contribution to Coptic revitalization, we also coupled THOTH AI with the music-generation system SUNO to produce Coptic songs. The authenticity of the generated lyrics is currently being formally evaluated by a panel of Coptologists and members of Coptic Orthodox liturgical communities, scoring orthography, morpho-syntactic acceptability, and stylistic appropriateness; preliminary results will be reported at the conference.
THOTH AI represents a measurable advance for the computational philology of Ancient Egyptian–Coptic, but is not a substitute for philological reasoning. Three limitations frame future work: dependence on a proprietary platform (we are piloting an open-source Langflow replication); the absence of a community benchmark for Coptic MT (under development with Coptic SCRIPTORIUM); and the inadequacy of surface-form metrics for literary translation, addressed by an in-progress rubric-based human evaluation protocol. By coupling LLM reasoning with rigorous philological data, THOTH AI shows that low-resource ancient languages can be analyzed with precision that materially supports—without replacing—traditional scholarship.