DH 2026

Daejeon, July 27–31

Poster

TiamaT: A Reflexive and Iterative Protocol for Visual Analysis in Digital Humanities

Marion Charpier
Ecole nartionale des chartes, France · marion.charpier@chartes.psl.eu

The application of computer vision to cultural heritage and historical images has expanded significantly in recent years, particularly for tasks such as object detection and visual classification, despite limitations related to visual variability, stylistic heterogeneity, and the scarcity of annotated training data. Recent work in Digital Humanities has demonstrated both the potential of deep learning for the analysis of medieval manuscripts and the methodological challenges posed by heterogeneous visual materials and limited training data (Aouinti et al. 2022). However, existing approaches often rely on workflows optimized for performance and scalability, offering limited support for interpretative control, critical evaluation, and epistemological transparency – issues that have been repeatedly highlighted in DH and visual studies. In response, recent best practices emphasize human-in-the-loop methods, careful dataset construction, and reflexive engagement with algorithmic outputs. Building on DH frameworks such as distant viewing, TiamaT proposes an iterative protocol that bridges large-scale computational image analysis with close, expert-driven interpretation (Arnold and Tilton 2023).

TiamaT is a modular protocol designed to support the automated description of complex historical images under the critical oversight of domain experts and humanities scholars, addressing a gap in existing pipelines that prioritize performance over interpretative control. Developed since 2023, it emerged from a collaboration between two DH projects – Digital Heraldry (medieval heraldic imagery) and HORAE (Books of Hours and religious iconography) -both confronting the limits of existing visual analysis pipelines when applied to heterogeneous, symbolically dense corpora such as coats of arms or devotional illustrations (Schneider and Hiltmann 2023; Daille et al. 2019). A third experimental case study, the Jean Royère collection (Musée des Arts décoratifs, Paris), further tested the protocol’s transferability across media, periods, and research questions (Bermès and Charpier, 2025). The shared goal is not full automation, but the construction of analyzable visual datasets that remain interpretable and available for discussion by peers.

TiamaT is structured as an iterative loop rather than a linear pipeline. The workflow comprises four interconnected stages: (1) the creation of ground truth through manual and semi-automated annotation; (2) model training, primarily using object detection architectures such as YOLOv11 (Jocher et al. 2023), combined with systematic data augmentation via Albumentations – for instance, perspective transforms applied to the Royère collection to simulate design variability; (3) inference on new or expanded datasets; and (4) correction and evaluation, where outputs – YOLO format annotations (txt files and relative xywh coordinates), confusion matrices, and Precision/Recall/F1 scores – are critically assessed and reintegrated into subsequent iterations, allowing both data and models to evolve in dialogue with research hypotheses. This approach aligns with recent heritage-focused computer vision studies that explore few-shot and data-efficient strategies adapted to small, specialized visual corpora (Ibrahim et al. 2022).

Annotation plays a central epistemic role within this process. Rather than being treated as a purely technical prerequisite, annotation decisions – such as category definition, visual delimitation, and granularity – are understood as interpretative acts that shape the resulting knowledge (Hodel 2022). Human validation is therefore embedded throughout the workflow, ensuring scholarly oversight of algorithmic behavior. This position echoes recent reflections on annotation practices in cultural heritage computer vision projects, which emphasize reflexivity, documentation, and critical engagement with dataset construction (Wuyts 2021).

TiamaT has been developed across diverse visual domains: medieval manuscripts, heraldic systems, religious iconography, and decorative arts. This diversity broadens the range of visual materials addressed by computer vision in DH and fosters interdisciplinary dialogue between technical methods and visual studies. As a reproducible, adaptable, and reflexive framework, TiamaT targets both researchers: researchers with a technical background, via Python scripts and Jupyter notebooks (github.com/Chaouabti/Tiamat), and humanities scholars without ML expertise, via a dedicated application, currently functional and in active development (github.com/TiamaT-app). This dual accessibility reflects TiamaT's broader ambition to lower the barriers to critically controlled computer vision workflows in DH research.

TiamaT is conceived as a living and shareable protocol, supported by progressive documentation (including a README, a recorded presentation, and a methodological paper). Ongoing work explores its extension to segmentation, fine-grained recognition, and multimodal approaches. TiamaT is also part of a broader methodological ecosystem: a companion tool for evaluating ontology robustness via inter-annotator agreement metrics (Mean IoU, Cohen's Kappa, confusion matrices) based on YOLO-format annotations (github.com/Chaouabti/ontology-robustness-eval) addresses the challenge of dataset quality beyond standard performance metrics. Together, these tools offer a reusable, interoperable, and sustainable infrastructure for critically controlled computer vision workflows in DH. The poster invites discussion on how such reflexive, human-centered approaches can foster more responsible and epistemologically robust uses of computer vision in Digital Humanities.

References
  1. Aouinti, Fouad, Victoria Eyharabide, Xavier Fresquet, and Frédéric Billiet. 2022. “Illumination Detection in IIIF Medieval Manuscripts Using Deep Learning.” Digital Medievalist 15 (1).https://doi.org/10.16995/dm.8073.
  2. Bermès, Emmanuelle et Marion Charpier. 2025. « Repenser les collections patrimoniales par le prisme de l’IA 2025 ». Communication présentée à Conférence nationale sur les applications de l’intelligence artificielle, Dijon, 30 juin-1er juillet. https://hal.science/hal-05138697v1
  3. Daille, Béatrice, Amir Hazem, Christopher Kermorvant, Martin Maarand, Marie-Laurence Bonhomme, Dominique Stutzmann, Jacob Currie et Christine Jacquin. 2019. « Transcription automatique et segmentation thématique de livres d’heures manuscrits. Traitement automatique des langues 60 (3) : 13-36. https://aclanthology.org/2019.tal-3.2/.
  4. Hodel, Tobias. 2022. « Die Maschine und die Geschichtswissenschaft : der Einfluss von Deep Learning auf eine Disziplin ». Dans Digital History. Konzepte, Methoden und Kritiken Digitaler Geschichtswissenschaft, édité par Karoline Dominika Döring, Stefan Haas, Mareike König et Jörg Wettlaufer, 65-80. Berlin : De Gruyter. https://doi.org/10.1515/9783110757101-004. 
  5. Ibrahim, Bekkouch Imad Eddine, Victoria Eyharabide, Valérie Le Page, and Frédéric Billiet. 2022. “Few-Shot Object Detection: Application to Medieval Musicological Studies.” Journal of Imaging 8 (2): 18.https://doi.org/10.3390/jimaging8020018.
  6. Jocher, Glenn, Jing Qiu, et Ayush Chaurasia. 2023. Ultralytics YOLO. Version 8.0.0. Ultralytics.https://github.com/ultralytics/ultralytics.
  7. Schneider, Philipp and Torsten Hiltmann. “The History of Heraldry Revisited. Introducing the Digital Heraldry Ontology to describe, contextualise, and analyse medieval and early modern coats of arms”. In Genealogica & Heraldica XXXV. Reformation, Revolution, Restauration. Proceedings of the Congress of Genealogical and Heraldic Sciences (Cambridge), herausgegeben von Paul A. Fox, Bd. 3. The Coat of Arms. Supplementary Volume. London, 2023, pp. 152-167 [PDF].
  8. Taylor, Arnold and Lauren Tilton. 2023. Distant Viewing: Computational Exploration of Digital Images. Cambridge, MA: MIT Press.https://doi.org/10.7551/mitpress/14046.001.0001.
  9. Wuyts, Jolan. 2021. “Annotating Datasets for Computer Vision to Recognise Architectural and Artistic Styles: Lessons from the V4Design Project.” Paper presented at Futurs fantastiques: The 3rd International Conference on Artificial Intelligence in Libraries, Archives and Museums, Paris, December 10. [Link].