Daejeon, July 27–31
Large-scale digitization initiatives at galleries, libraries, archives, and museums (GLAM institutions) have made millions of cultural heritage materials widely available via the Internet (Rosenzweig 2003, Solberg 2012, Ramsay 2014, Putnam 2016). This has transformed digitized cultural heritage collections into computationally tractable datasets, opening possibilities for algorithmic analysis and pattern recognition at scales previously impossible (Padilla et al. 2019, Mahey et al. 2019, Warren et al. 2021, Lee et al. 2020). But GLAM institutions do not consistently record dimensional metadata (an object's physical size) in machine-readable formats (Crane 2021, Fraga et al. 2024), creating barriers to computationally analyzing the material properties of cultural heritage objects at scale.
Physical dimensions convey important information about an object's creation, use, and cultural significance (Prown 1982, Deetz 1996). For example, a miniature portrait communicates different messages than a life-sized painting (Barratt 2003). Although human viewers can often infer dimensions from visual inspection or textual descriptions, computational systems struggle with this task. This paper presents a novel method for automatically calculating object-level dimensional metadata by applying computer vision (CV) to the outputs of existing standardized digitization workflows.
During digitization, GLAM institutions typically follow best practices established by the Federal Agencies Digital Guidelines Initiative (FADGI 2023) by photographing collection materials alongside digitization targets (also known as reference targets or calibration targets). Targets are physical objects included in the image for imaging calibration, color correction, and quality control. Crucially, each target has known, consistent physical dimensions. Although a target's size is visually evident to human viewers, it is computationally inaccessible.
Recent advances in CV demonstrate the potential for extracting structured information from cultural heritage images (Fiorucci et al. 2020). Machine learning has been successfully applied to cultural heritage imaging tasks, such as newspaper content classification (Lee et al. 2020) and typographical analysis (Warren et al. 2021). But digitization targets have not been treated as sources of computable metadata. We are using machine learning to identify these targets in object images, then use the targets' known dimensions to calculate a given image's pixel-to-centimeter ratio and determine the photographed object's physical size. Our work provides a method for unlocking the latent dimensional metadata already present in millions of digitized images.
Specifically, we are using machine learning to develop a custom CV model that will recognize and classify digitization targets across a heterogeneous collection of approximately 250,000 images from the Huntington Digital Library (HDL). This collection encompasses diverse media (including books, manuscripts, newspapers, maps, letters, postcards, blueprints, posters, and ephemera) digitized over multiple years using different imaging systems and protocols. This heterogeneity makes HDL ideal for developing a robust, generalizable model. Our computational challenge is distinguishing between approximately 18 different target types and variants, some of which closely resemble each other. (For example, GoldenThread targets come in three different physical sizes which share similar visual patterns.) We earlier trained a CV model capable of reliably classifying HDL three target types, confirming viability; our current work expands to all HDL target types.
We developed a scale bar target classifier by applying transfer learning on the EfficientNet-b0, a CNN model pretrained on the ImageNet dataset. This approach leverages a model trained on millions of general images for feature extraction, and we fine-tune the model using our scale bar dataset for our purposes of target classification, dramatically reducing data requirements and overfitting, compared to training from scratch. We begin data collection by randomly sampling, cropping, and manually labeling ~1,500 images. The controlled photography conditions typical of professional digitization workflows target types and our usage of a cropped target meant our dataset has low intra-class variance (a given type of scale bar will look basically the same in different archival images) than in other domains. Given a highly imbalanced dataset (varying 3-200+ raw samples per class) and low intra-class variation, we mitigated overfitting by applying class-aware data augmentation (rotations, brightness variation, translation, noise) to generate 200-500 training images per class. After training the model using this augmented dataset, we identified the most uncertain predictions, edge cases, and frequent target class confusions; then applied additional labeling and data augmentation efforts to ensure the training dataset better reflected the true population of target images. The resulting model reached a 97%+ accuracy on test dataset unseen in training. We will report aggregate and per-target accuracy metrics.
After training, our technical pipeline is as follows: the trained model identifies cultural heritage objects and digitization targets in unlabeled images. It determines pixel dimensions for the object and the target, then labels the target by type. We cross-reference the target's pixel dimensions with known physical measurements for the corresponding target type, then use these to calculate the overall image's pixel-to-centimeter ratio, and finally compute the physical dimensions for the photographed object. This process aims to support reliable metadata enrichment at scale for digitization efforts.
We thus enable the batch-processing of enriched metadata for legacy digitized images, with potential to dramatically improve availability and consistency of dimensional metadata in cultural heritage collections. Our documentation and open-source code will enable other institutions to apply this approach to their collections.
This research goes beyond the current state of best practices by moving metadata generation from a manual, labor-intensive entry process to an automated, scalable workflow. GLAM institutions have already invested heavily in high-quality digitization following professional standards such as FADGI (2023). Our work demonstrates how existing digitization infrastructure can be computationally leveraged for metadata enrichment without requiring additional imaging or changes to current workflows. This approach is particularly valuable for institutions with limited cataloging resources, enabling them to enrich legacy digitized materials at scale. This represents a key step toward realizing the possibilities of "collections as data" by transforming visual information into structured data, enhancing both the accessibility and the analytical potential of digitized cultural heritage collections.
Beyond metadata enrichment, this research enables new forms of scholarship and public engagement. For example, scholars could trace how the size of printed media (like maps, pamphlets, or newspapers) changes across time and place, thereby linking physical form to production technologies, circulation networks, and embodied information practices. This approach is particularly valuable for collections where physical format carries cultural significance—from artists' books and zines, to Indigenous material culture where size and scale can carry ceremonial meaning. Additionally, accurate dimensional data facilitates the creation of life-size virtual objects, ensuring that digital surrogates communicate the material characteristics of physical originals (Fraga et al. 2024). These applications—from computational analysis of media formats to immersive virtual experiences—show how making physical dimensions computationally accessible can bridge the gap between material culture and digital scholarship.