The digital turn in historical inquiry (Graham et al. 2015) has long promised to bridge the gap between “distant reading” (Moretti 2013) of massive corpora and the “close reading” (Richards 1929) required for nuanced interpretation. However, for historians dealing with early modern private correspondence (Daybell 2012; Gao forthcoming), this promise often hits a technological wall. These documents are characterized by their “small data” (Kitchin / Lauriault 2015) nature—specialized, highly contextual, and varying in volume—and, crucially, their paleographic and linguistic diversity. When the corpus spans the divide between the Latin alphabet and the logographic or syllabic scripts of major East Asia languages (Korean, Chinese and Japanese), the “black box” of commercial, proprietary Artificial Intelligence often fails to provide the transparency or linguistic dexterity required for rigorous scholarship.

This paper reports on a series of experiments designed to structure the analysis of early modern letters (c. 1500–1850) using Local Large Language Models (LLMs) in service of exploring how people experienced separation (Daybell 2025). Our project engages directly with two key themes of the conference: Annotating, by utilizing AI to structure unstructured historical text, and Translating, by interrogating the translingual capabilities of AI across disparate scripts. We focus on a dataset covering four linguistic contexts—Sweden, England, Korea, and Japan—and five scripts: the Latin alphabet, kanji, hangul, hanja, hiragana, and katakana.

We argue that while generalist “multilingual” models often exhibit an Anglocentric bias that flattens non-Western texts, the strategic use of language-specific local models can effectively democratize the annotation process. By treating the LLM not as an oracle but as a heuristic tool—a filtering assistant—historians can scale their analysis without losing control over privacy and interpretation.

The experiments utilized a corpus of approximately 1,500 digitalized letters, representing a subset of a larger ongoing project Moved Apart . The distribution is uneven, ranging from roughly 20 letters in the Japanese corpus to over 700 in the English corpus, reflecting the realities of archival preservation and digitization.

We have five researchers in charge of different linguistic corpora of letters, each overseeing collections ranging from several hundred to several thousand items.

The material consists of private correspondence, primarily familial interactions among the well-off, transcribed via a mix of Optical Character Recognition (OCR), Handwritten Text Recognition (HTR), and manual transcription.

A central tenet of our methodology is the rejection of commercial APIs in favor of Local LLMs. This choice is driven by two factors: Privacy and Reproducibility. Private correspondence, even from the early modern period, often resides in sensitive archives or represents cultural heritage data that should not be fed into opaque commercial training sets (Shamberger 2018). Furthermore, relying on open-weight models ensures that our experiments can be reproduced (Ries et al. 2023) by other scholars without risk of model versions changing overnight. All our experiments were conducted on consumer-grade hardware (M4 MacBook Pro) using Ollama as the inference runtime, with models quantized to fit in memory.

Keyword search is an established and trusted technique for detecting and exploring phenomena in digitalized corpora. However, it only works with a clearly defined vocabulary of keywords – which even if we could define one such a list for one of the languages presupposes that we knew, more or less, exactly what we were looking for before we began. Since expressions that indicate how people experience separations could be formulated anywhere between explicit statements to indirect, veiled hints – making such a list is a near impossible task. In contrast, language models can be very adept at dealing with nuance and ambiguity. Under the theme of Annotating | Beyond Patterns, we tasked the models with structuring the unstructured text of the letters.

We adopted prompt engineering (Sahoo et al. 2024) rather than fine-tuning — whether supervised, preference-based, or through weak supervision. The reasons are both epistemological and practical. The phenomena under study — how people experienced separation — are ephemeral and resist operationalization into fixed categories: expressions range from explicit statements to indirect, culturally embedded hints, making traditional ground truth (Liang et al. 2016) not merely expensive but conceptually fraught (Rosenwein 2015). Fine-tuning of any kind presupposes a stable task definition, which we had not yet established. Practically, six weeks of technical development time and consumer-grade hardware made iterative prompting the only feasible approach for testing across four languages and multiple models.

Instead of ground truth we relied on qualitative group evaluation (Karjus 2025) sessions where domain experts (historians specializing in the respective languages) assessed model output qualitatively — rather than against a fixed benchmark — together with the entire research team. This combination of prompt-engineering and group evaluation served as a dialogue between researchers and models: Rather than assuming a fixed capability, we adjusted our prompts, and our expectations, based on model behavior — acknowledging that effective annotation is not a one-way extraction but a negotiation between the historian’s questions and the model’s limitations. The discussions and reflections from these meetings helped us decide which prompts to adjust, which to keep and which models and prompts to abandon. We tested prompts targeting multiple variables — emotional intensity rankings, relational networks, and mentions of distance — before converging through group evaluation on a terse, four-line prompt with equivalent prompts were written in Japanese and Korean for the language-specific models, with structured output enforced via schema validation rather than free-text generation.

Consistent with known limitations of multilingual models, we observed a significant divergence in performance based on script, directly addressing the theme of Translinguality in the Era of AI. Initial tests with generalist, instruction-tuned open-weight models

Gemma3 27B, Llama 3.3 70B, and Mistral Small 3.1 24B — all instruction-tuned, run via Ollama with models quantized to fit in memory.

handled English and Swedish texts competently but, as expected, struggled with Japanese and Korean. While the models could parse the tokens, their ability to “understand” and perform even simple extraction tasks degraded sharply. We observed instances of “token loops,” where the model would output repeating nonsensical character sequences when the context window (approx. 120k tokens) was not the issue, but rather the model’s inability to navigate the logographic density of kanji or the agglutinative nature of Korean.

Furthermore, we encountered distinct Anglocentric hallucinations (Joshi et al. 2020; Cao et al. 2023). When asked to extract locations from East Asian texts, generalist models would occasionally hallucinate Western capitals (e.g., identifying “London” in a Korean text where it did not exist) or attempt to translate concepts rather than analyze them. The models displayed a bias toward English reasoning; even when prompted in the source language, the internal logic appeared to map non-Western concepts onto Western structural equivalents. We pivoted to language-specific, instruction-tuned models

EXAONE 3.5 32B (LG AI Research) for Korean; Linkbricks Horizon AI Japanese Superb V3 70B for Japanese — among several tested and discarded alternatives including DeepSeek-R1-Bllossom 70B, Trillion 7B, and SEOKDONG-llama3.1.

for Japanese and Korean. This hybrid approach dramatically improved performance. It suggests that in the “contact zone” between humans and machines, a “one-size-fits-all” multilingual model is often insufficient for historical inquiry outside the Anglo-European sphere.

A recurring obstacle was the models’ tendency toward over-interpretation (cf. sycophancy in the LLM literature (Perez et al. 2023)). Standard epistolary formulas (Bergs 2004) (e.g., “Dear Sir,” or standard honorifics) were frequently interpreted as evidence of deep emotional intimacy. However, contrary to expectations, the models handled archaic spelling and early modern grammar with remarkable resilience. The noise introduced by OCR/HTR errors (Cordell 2017) or non-standardized early modern spelling did not significantly hamper the models’ ability to parse meaning. In fact, the models would occasionally silently correct these spellings in the output—a behavior that is technically a hallucination (Ji et al. 2023) but functionally useful for data structuring.

A further, rather counter-intuitive but fortuitous observation was that the models seemed to thrive on less instruction, not more. Earlier iterations used longer, more prescriptive prompts in English — including worked examples — which paradoxically encouraged the models to hallucinate finding those examples. Furthermore, having to create a long list of possible ways to express what we were looking for would, at best, bias the models towards excluding alternative, but highly relevant expressions. When we instead gave the models the terse four-line prompt described above — without definitions or examples — it gave them freedom of interpretation. We found that they rarely missed significant elements and the false positives are often very obviously wrong. This combination renders the Local LLM a highly effective filtering tool. It allows the historian to query the dataset to find the “50 letters out of 1,500” likely to contain specific emotional patterns or relational dynamics, vastly increasing the efficiency of the research process.

Our experience challenges the binary often presented in AI discourse between “Big Data” and “Small Data.” While 1,500 letters constitute “Small Data” in the context of machine learning, they represent a significant cognitive load for a single researcher. Local LLMs bridge this gap, serving as interpreters that can handle the volume of the archive while respecting the “smallness” (privacy and context) of the individual letter.

The experiments demonstrate that the utility of AI in history is not dependent on the model achieving human-level nuance. Even a “frozen,” imperfect model running on a laptop can reveal structural patterns—provided the historian treats the output with a “trust but verify” mindset. The primary risk identified is not that the model fails, but that it succeeds too well at projecting Western structural expectations onto non-Western materials. In the presentation, we will demonstrate concrete examples of how the filtering process surfaced historically significant expressions of separation across the four linguistic corpora.

This report outlines an iterative and agile, yet reproducible approach to using Local LLMs for annotating multi-script historical correspondence. We conclude that while generalist AI models struggle with the translingual demands of global history, a curated approach using language-specific local models can unlock new methods of analysis. By having the machine structure unstructured corpora, we wish to augment the historian (Shamsieva et al. 2026) by providing high-level mappings. Nevertheless, we must remain vigilant against the Anglocentric and other biases inherent in these tools. The future of AI in Digital Humanities lies not in massive, opaque commercial models, but in the localized, iterative dialogue between the scholar and the open-source models.