Daejeon, July 27–31
Scholars have long classified narratives and discourses by grouping similar plots, motifs, and themes. The Aarne–Thompson–Uther Index is a representative example of this tradition, classifying folktales by recurrent tale types and motifs (Uther 2004). Yet such work relied on expert manual comparison, which limited sample size and left fewer opportunities for systematic quantitative validation. Today, computational methods allow much larger-scale textual exploration.
In Digital Humanities, topic modeling has become a method for examining large-scale discourse (Blei et al. 2003; Grootendorst 2022; Meeks / Weingart 2012). This approach is particularly relevant to social media data, where large volumes of user-generated texts make manual reading difficult. Social media comments record contemporary emotions, participation, and cultural response (Thelwall 2018), but they are also heterogeneous and context-dependent, requiring interpretation rather than mechanical analysis (Grimmer / Stewart 2013; Marwick / Boyd 2011; Hine 2000). Topic modeling is thus exploratory, not a substitute for interpretation.
Previous studies have compared LDA and Top2Vec to evaluate topic model performance (Egger / Yu 2022). Recently, BERTopic has become prominent for producing context-aware clusters in short-text corpora (Kaur / Wallace 2024; Wang et al. 2024), leading to pipeline-optimization studies (Azher et al. 2024; Borčin / Jose 2024). However, statistical metrics used to evaluate topic models, such as coherence and diversity, do not necessarily make topics interpretable (Chang et al. 2009; Röder et al. 2015). Scholars have also noted that automated content analysis should be validated on each new dataset, since its performance can vary by context (Grimmer / Stewart 2013). This issue is especially relevant to social media texts, which need to be read in relation to the social, cultural, and communicative conditions in which they are produced (Roberts et al. 2014: 1068; Kozinets 2002: 65).
Content analysis emphasizes manifest and latent meanings (Berelson 1952), while thematic analysis identifies, reviews, and defines themes in relation to research questions (Braun / Clarke 2006). Unlike automated clustering, human interpretation can account for context, implication, symbolism, and subtle semantic differences. It can also reorganize model-generated topics into themes suited to the analytical purpose. Thus, human thematic interpretation is needed not only to validate topic models, but also to turn exploratory outputs into humanities research.
Topic and model selection have been examined in traditional models (Griffiths / Steyvers 2004; Taddy 2012), yet less is known about BERTopic in multilingual and multicultural datasets. It remains unclear how cross-linguistic preprocessing choices alter topics, how they should be decided, and when multilingual topics are comparable. This study addresses this gap by analyzing YouTube comments from Korea and the United States through BERTopic and human interpretation. It quantitatively demonstrates the usefulness of human thematic interpretation for data, and proposes a workflow prioritizing interpretability, transparency, and contextual validity over statistical stability alone.
Focusing on the cultural nuances of South Korea and the United States meditation motivations, we analyzed comments from guided meditation videos on YouTube. To ensure cross-cultural comparability, the research design prioritizes a balanced framework that aligns disparate linguistic and communicative patterns into a singular interpretive structure. In accordance with the ethical guidelines of the Association of Internet Researchers (AoIR 2020), all personal identifiable information was anonymized, and quoted comments were modified and paraphrased to protect user privacy. A total of 56,767 comments were collected from U.S.-based YouTube channels and 41,279 comments from South Korean channels. To prevent the overrepresentation of highly active users, we limited the data to one comment per user. For users with multiple entries, the comment with the highest like count was retained; if like counts were identical, the longer comment was selected to maximize information density.
Text preprocessing is a critical factor in unsupervised text analysis (Denny / Spirling 2018; Schofield et al. 2017). After removing advertisements, spam, URLs, emojis, and mentions, we normalized whitespace and repeated characters. We then labeled comments as short, medium, long, or extremely long according to each language’s length distribution. Short reactive comments and extremely long outliers—up to 8k characters—were excluded, since short social media texts often lack sufficient co-occurrence and contextual information (Mehrotra et al. 2013; Churchill / Singh 2021). Medium-length comments were retained with minimal intervention, while long comments were segmented into sentences to improve thematic extraction.
We then modeled topics with BERTopic, which identifies thematic structures through transformer-based contextual embeddings, dimensionality reduction, and clustering. Contextualized document representations produce more coherent topics than traditional bag-of-words approaches (Bianchi et al. 2021). We used a multi-stage refinement procedure: initial topic extraction, removal of irrelevant or noise topics, and thematic aggregation (Cheng et al. 2022). For cross-cultural comparison, multilingual embeddings served as the main strategy; to check possible English-centric bias in multilingual models (Papadimitriou et al. 2023), each language was also modeled independently with language-specific embeddings. In HDBSCAN clustering, min_cluster_size was adjusted by document volume to avoid fragmented, uninterpretable micro-clusters (McInnes et al. 2017), and domain-specific stopword sets were matched across languages to ensure consistent topic representation.
We systematically compared five model configurations: 1) the primary-stage model, 2) the secondary-stage model, and three third-stage variants: 3) human-guided aggregation, 4) automated hierarchical aggregation, and 5) machine-led aggregation constrained to the number of thematic clusters identified through human-guided aggregation. Since topic granularity remains a central issue in topic modeling (Taddy 2012), these configurations were assessed using topic coherence, diversity, and qualitative interpretability.
Analysis of the collected data showed distinct participation patterns: the U.S. dataset contained 46,107 unique users among 56,767 comments, whereas the South Korean dataset contained 22,557 among 41,279, indicating a higher concentration of heavy users in Korea. For length-based labeling, we used character counts rather than word counts because inconsistent spacing in Korean could distort token- or word-based estimates. Following information-theoretic studies showing that Korean syllables carry higher information density than English characters (Han et al. 1996; Pellegrino et al. 2011), percentile-based thresholds were adjusted by language.
Short comments were more common in the U.S. data and were excluded because they offered limited insight into meditation motivations. Extremely long comments were treated as emotional venting, more suitable for qualitative inquiry than automated modeling. The final corpus contained 40,905 U.S. documents and 24,169 Korean documents: 30,790 medium and 10,115 long comments in English, and 17,424 medium and 6,745 long comments in Korean.
As shown in Figure 1 and 2, the automated hierarchical aggregation in stage 3 achieved the highest coherence score, but compressed 19 topics into 4 groups, reducing thematic detail. This shows a key limitation of automated aggregation: although it efficiently merges topics close in lexical or embedding space, it may miss interpretive distinctions. By contrast, human-led classifications such as the Aarne–Thompson–Uther Index distinguish meaningful units such as tale types, motifs, actions, characters, and settings. Likewise, our human aggregation examined whether machine-generated topics should remain separate or be regrouped by contextual meaning.
For comparison, we evaluated automated and human-led aggregations at the same number of grouped topics. The human grouping was built through qualitative coding: experts identified eight themes, two independent coders assigned each topic to them, and intercoder reliability was measured. Under this condition, human aggregation showed higher coherence and lower diversity. This suggests that it did not arbitrarily reduce topics into themes, but systematically regrouped them by semantic distinctions that automated clustering could miss.
As supplementary validation, we also modeled each language corpus with language-specific embeddings: all-mpnet-base-v2 for English and snunlp/KR-SBERT for Korean. This allowed us to check whether comparable themes appeared across embedding strategies and whether the topics reflected culturally meaningful patterns rather than artifacts of multilingual embeddings.
Due to the informal and fragmented nature of YouTube comments (Thelwall 2018), many documents were assigned to the -1 category. We therefore re-examined them to separate noise from ambiguous or overlapping themes, modeled the outliers to detect residual thematic structure, and tested a classifier trained on examples from the eight themes to identify comments spanning multiple themes.
This study contributes to Digital Humanities by examining how multilingual and multicultural corpora can be modeled, compared, and interpreted through BERTopic. It treats topic modeling not as a fully automated pipeline, but as an interpretive aid that requires continuous researcher intervention. To enable cross-linguistic comparison, the main analysis uses multilingual embeddings; to check possible English-centered bias, language-specific embeddings are used in sub-corpus analyses. The results show that linguistic structure and information density must be considered from preprocessing to evaluation. Most importantly, the study demonstrates that human-guided thematic aggregation can add interpretive value beyond automated clustering by distinguishing semantic differences among seemingly similar expressions and reorganizing topics according to contextual and cultural meaning. Finally, unclassified documents should not be dismissed as noise, since they may contain ambiguous, overlapping, or culturally nuanced discourse.