Emotions are a fundamental aspect of human experience, serving as a natural response to reality for both individuals and social groups. For this reason, determining the emotional valence, or sentiment, of a text using computational linguistics methods has been actively pursued over the past two decades as a primary or auxiliary task across a wide range of studies. This task frequently appears in applied natural language processing research, particularly in the analysis of product and service reviews (Mutinda et al. 2023), as well as social media comments (Mehra 2023) and posts (Schmidt et al. 2022), in order to detect emotions in real time. A distinct line of research focuses on the affective dimension of literary fiction, ranging from associating texts with specific emotions (Jacobs 2019) and constructing emotional arcs (Reagan et al. 2016) to modelling sentiment interactions between characters (Kim / Klinger 2019).

Methodologically, sentiment analysis and emotion detection typically rely on two approaches: lexicon-based and machine learning-based. The lexicon-based approach involves compiling a dictionary of evaluative vocabulary and defining rules (linguistic, contextual, or logical) to determine the polarity of specific text fragments (Lukashevich 2021). Lexicons may be specifically designed for a particular language and text type, expanded to include domain-specific features, or adapted through automatic translation from other languages. Machine learning methods, in turn, frame sentiment detection as a classification task (e.g., Support Vector Machines (SVM), Naive Bayes classifiers, logistic regression, and gradient boosting) (Nguyen et al. 2018). Such algorithms require the creation of suitable training datasets.

In contrast to other domains, resources for analysing literary texts are limited, with prominent dictionaries and datasets for Russian primarily derived from non-fiction. Key resources include PolSentiLex (Koltsova et al. 2016), RuSentiLex (Loukachevitch / Levchik 2016), and the translated NRC Word-Emotion Association Lexicon (Mohammad / Turney 2013), whereas the largest available datasets are CEDR (Sboev et al. 2021), RuIzardEmotions (https://huggingface.co/datasets/Djacon/ru-izard-emotions), and the translated RuGoEmotions ( https://huggingface.co/datasets/seara/ru_go_emotions).

While combining approaches can improve robustness (Yadav et al. 2020), recent works also explore the use of generative models for emotion annotation, presenting them as a way to overcome domain-specific limitations (Borst et al. 2023). However, several issues remain unresolved, including discrepancies between sentiment scores and the ratings assigned by users in reviews and comments (Maks / Vossen 2013), as well as between automatic predictions and human judgments (Rebora et al. 2023; Sherstinova et al. 2023). This raises a key methodological question: when evaluating a text, do we model the emotionality embedded in it or the emotional response perceived by a reader? The problem becomes particularly relevant in the context of fiction, where interpretation is shaped by readers’ social, cultural, and professional experiences. It is further complicated by the typically greater length of literary texts compared to reviews, as well as by their figurative nature, which invites varied interpretations.

In this paper, we present an annotated dataset based on 240 Russian short stories written between 1901 and 2000 (Sherstinova et al. 2023), with a total volume of 531,725 words. The first level of annotation targets lexical emotion markers: words are tagged according to 11 emotion categories (fear, sadness, anger, disgust, shame, happiness, pleasure, amusement, surprise, pride, relief), as well as two additional tags for general positive and negative sentiment. Each text was analysed independently by two annotators with backgrounds in philology. Annotators were instructed to mark words that express specific emotions, have clear positive or negative meanings, or carry strong emotional connotations affecting the overall mood of the text. In total, 75,742 lexical units were labelled.

The second level captures reader perception, with annotators evaluating 5,932 story fragments using valence and arousal scales (Moskvina / Kirina 2026). Texts were segmented to preserve structural coherence, allowing for nuanced assessments of the emotional background. Valence was measured on a scale from –3 to +3 (from the most negative to the most positive), and arousal on a scale from 1 to 5 (indicating emotional intensity). This approach makes it possible to assess the overall emotional impact of each fragment on the reader.

Preliminary analysis reveals several patterns in the representation of emotions in 20th-century short prose. Sentiment vocabulary shows a predominance of negative over positive items (35.27% vs. 29.16%). Among specific emotions, sadness (7.40%), fear (6.22%), happiness (4.98%), and anger (4.93%) are the most frequent. Aggregated valence scores indicate that 67.92% of stories are perceived as predominantly negative, with relatively little variation across the period (σ = 29.81, calculated at the story level). These findings do not support the positivity bias observed in English-language corpora, namely the predominance of positive lexis across texts of different genres, often interpreted as reflecting “a preference of humankind for pro-social and benevolent communication” (Warriner et al. 2013). We also report a moderate negative correlation between mean valence and arousal (r = –0.33), suggesting that texts with high arousal are not necessarily positive, and vice versa.

We argue that the proposed two-level annotation framework enables a multidimensional investigation of textual emotionality, particularly in fiction. Exploratory analysis offers several insights into the literature of the period under consideration, while validation on a larger corpus may help establish the stability of emotional patterns in texts and enhance our understanding of their variability in response to sociocultural and historical changes. While the primary goal of creating the dataset was to support reliable research on domain-specific texts, the results presented can also serve as a reference point for evaluating sentiment analysis tools designed for processing Russian-language texts. This dataset also offers potential for extending existing emotion lexicons and for adapting the proposed approach to other languages and literary genres. Additionally, it can aid in fine-tuning models, including those used for AI-based annotation.

Acknowledgements

This work is an output of a research project HSE-BR-2025-032 implemented as part of the Basic Research Project at HSE University.