Daejeon, July 27–31
Many existing tools for computational literary studies mostly focus on a single language, and the annotation methodologies behind them often lack transparent, detailed documentation, making their standards and handling of edge cases unclear. As Fiormonte (2021) and Viola & Spence (2023) have argued, the dominance of English in digital humanities leads not only to uneven language coverage but also to English-centric assumptions embedded in annotation categories and research questions. Conventional annotation projects often develop guidelines in a single language (typically English) and later translate or adapt them to others (e.g., Minutolo et al., 2022). This approach risks embedding English-centric assumptions. To address this gap—both the linguistic bias and the opacity in annotation practice—we introduce a multilingual annotation workflow focused on characters in stories, together with a full documentation. We detail not only the resulting guidelines but, centrally, our process, that includes the iterative, discussion-driven refinement of annotation rules with native speakers across seven languages (Chinese, Dutch, English, Indonesian, Italian, Korean, and Spanish) as well as how to resolve cross-linguistic inconsistencies, handle edge cases, and document decision trails. While we report on the resulting dataset and guidelines, the primary contribution of this paper is this collaborative methodology that actively engages annotators, moving beyond an academia-centric approach to create a more consistent resource for computational literary studies.
Characters are central to both traditional and computational literary studies, they serve as focal points for readers’ engagement, driving narrative progression and thematic exploration (Eder et al., 2010). In computational studies, character analysis has evolved from manual annotation to automated recognition, with methods including entity recognition, character modeling, and interaction analysis, with an increasing demand for high quality annotated dataset (Jockers, 2013; Piper, 2024).
A foundational development in this field is the BookNLP https://github.com/booknlp/booknlp
Beyond coreference, understanding dialogue and social relationships is crucial for narrative analysis. Recent work includes the SIG (Speaker Identification via prompt-based Generation) approach by Su et al. (2024), which uses generative prompts to identify speakers in the English Project Dialogism Novel Corpus (PDNC), offering a flexible alternative to traditional classification models. Other contributions include datasets for annotating dynamic social relationships (Massey et al., 2015) and frameworks for characterizing interaction types (Piper et al., 2024).
Despite these advances, a significant bias persists: the majority of robust NLP resources for literary analysis are designed for English. Efforts for other languages are often built on limited literary data or address specific sub-tasks. For instance, Italian resources have been developed using coreference resolution corpora like OntoNotes (Minutolo et al., 2022) and neural systems like OntoCorefIT (Guarasci et al., 2021). For Germany, the LitBERT https://www.informatik.uni-wuerzburg.de/datascience/projects/nlp/litbert/
We created a multilingual fiction corpus of around 100,000 words per seven different languages (Chinese, Dutch, English, Indonesian, Italian, Korean, and Spanish) for human annotation to support next-step automatic tagging, character analysis, and cross-cultural comparative studies. The corpus includes stories from the Archive of Our Own (AO3) fanfiction platform and stories from platforms like Wattpad, Postype, and Lofter.
Our data selection follows strict ethical and practical sampling criteria. We selected stories out-of-copyright or for which we received the explicit permission of the authors. Works were sampled with length from 200 to 15,000 words and metadata were used to ensure diverse representation of fandom, genre, and narrative style, balancing the quality of work for human annotation with efficiency for model training. Stories with explicit sexual depictions were excluded.
The character annotation guideline comprises four tasks:
To conduct the four tasks, Figure 1 shows the workflow of our annotation work, for the multilingual nature of the project, two native-speaking annotators were recruited for each of the seven languages. Since the guidelines were primarily developed based on English, Italian and Chinese, weekly meetings were held during the first two months of annotation to discuss and clarify rules, allowing the guideline to be iteratively refined by incorporating language-specific cases and rules, ensuring cross-lingual consistency (Chai & Strube, 2023). We choose INCEpTION https://inception-project.github.io/
Figure 1. Annotation Workflow
Our annotations are focused on characters, including all animate mentions; all mentions of inanimate entities are excluded.
For Task 1, our annotation framework aligns with the objectives of previous projects (Bamman et al., 2020; Martinelli et al., 2025), but introduces extensions in scope and granularity for its multilingual design and specialized focus on fanfiction. The guideline specifies a typology of character mentions, including zero pronouns, which are prevalent in pro-drop languages such as Spanish, Italian, and Chinese. For example, in the Spanish sentence “Cerró la puerta” ([He] closed the door), the omitted subject is annotated using a zero-width span, addressing a gap in prior resources and enhancing cross-lingual consistency (Aloraini & Poesio, 2020). More results and language-specific challenges analysis are reported in van Cranenburgh et al. (2026) The supplementary materials can be found in the repository: https://github.com/GOLEM-lab/GOLEMcoref/
The annotation criteria were established and polished based on iteration discussion with annotators, below Table 1 shows the examples raised by the annotator and our corresponding clarification. This documentation process not only ensures transparency and reproducibility in our annotation methodology, but also strengthens the guideline through practical validation and language-specific adaptation, resulting in a more consistent and reliable multilingual dataset for future research in character analysis and narrative understanding.
Table 1. Q&A summary and annotation guideline refinement of Task 1 (part)
In Task 2, for each dialogue annotation, we specify the Speaker and Addressee, linking them to previously annotated CharacterMention annotations. The addressee is defined as the specific intended recipient, requiring inference based on context if not explicitly mentioned. Annotators reported many instances where the addressee was not clearly inferable from the local context, necessitating the “Unknown” label and a comment, underscoring the limitations of relying purely on surface patterns in dialogue.
Table 2. Q&A summary and annotation guideline refinement of Task 2 (part)
Task 3 focuses on classifying mentions of Character Properties into four categories: Physical Features, Personality, Emotion/mental state, and Biography. Annotators only label the relevant span in the text with one or more of these categories, without specifying the property value (e.g. hair color) because this would be too demanding for the annotation task.
A critical aspect of the guidelines is defining the boundaries of human inference to ensure high-quality data that retains interpretive integrity. We provide clear distinctions. For example, Annotators are explicitly instructed not to interpret from Emotion/mental state to Personality. For instance, annotating “was relieved” as Emotion/mental state does not necessitate a further interpretation that the character has a “caring” Personality. Similarly, inference from a physical description (e.g., “kind face”) to an inner trait (“kind personality”) is also not allowed. These careful calibration of allowed and disallowed inference helps our annotator better make decisions.
Table 3. Q&A summary and annotation guideline refinement of Task 3 (part)
In Task 4, social relations are defined as directed, flowing from a Source character to a Target character (e.g., Source (A) loves Target (B)). The Relationship Type is a free-text input (e.g., mother, friend, lover). Annotators are required to label relationships when clearly presented or relatively clearly inferred through interaction, particularly for main characters. This open-ended approach to relationship typing captures the specific, often non-canonical, relationship nomenclature common in fanfiction. Compared to the other four tasks, as Table 4 shows, there are relatively fewer doubts from the annotators.
Table 4. Q&A summary and annotation guideline refinement of Task 4
The documentation related to this annotation process offers a methodology on how to engage the annotators for a high-quality, multilingual corpus to support multilingual cross-cultural literature study. This practice aims towards a robust multilingual resource that captures the nuances of characterization process, as well as capturing the discussion in the annotation process, thereby opening the data to richer, interpretation-driven literature research.