DH 2026

Daejeon, July 27–31

Wed, July 2914:00–15:30S053101-102
Short Paper

Describing the Third Dimension: A Schema and MLLM-based Metadata Generation Methodology for Historical Stereoscopic Photographs

Dhruva Gowda-Storz
EPFL, Switzerland · dhruva.gowdastorz@epfl.ch
Sarah Kenderdine
EPFL, Switzerland · sarah.kenderdine@epfl.ch

Introduction

Stereoscopic photography was the first 3D immersive mass medium, a dominant visual paradigm of the 19th and early 20th centuries, and a forerunner to today’s virtual reality. During seven decades of immense popularity, an estimated seven million stereographs were produced (Darrah 1977). Today, much of this material survives in archival collections, but the immersive experience is inaccessible through 2D digital scans (Figure 1). In the era of Virtual Reality, there have increasingly been efforts to computationally reactivate this material in archives and museums (Heineman 2012; Bethge et al. 2025; Gowda-Storz et al. 2025), which involves converting scans to 3D-ready digital formats by extracting the left and right images (Luz / Peixoto 2015). When scaled through crowd-sourcing (Vershbow 2013) or automatic pipelines (Gowda-Storz / Kenderdine 2025), this process holds immense potential to revive a forgotten immersive medium.

Figure A typical Underwood & Underwood stereo card with caption, series number, publisher info, and viewing instructions. Image courtesy of the author.

Yet, creating 3D-ready stereo pairs is a double-edged sword, as we discard the surrounding frame that functions not only as physical support but as a vehicle for the stereograph’s intrinsic metadata. Many frames contain captions, series numbers, publisher information, or even extended descriptions on the verso. The frame is as crucial a part of experiencing a stereograph as the images, situating views in space and time, providing historical context, or positioning individual cards within curated series. If the frame is separated, existing archival metadata is not always reliable enough to recover lost information, as metadata standards are not designed to be backups. It is unlikely that transcribed text from the frame would be included in metadata. Though some standards can capture this information, there is no guarantee that it will be recorded. Surveying 25 institutional collections, Goldberg / Clemons (2010) found that stereograph metadata is inconsistent and intrinsic features such as series statements are often omitted.

To address this gap, we propose an experimental metadata schema for archival stereographs, as well as a Multimodal Large Language Model (MLLM) based framework to populate it automatically at scale, complementing stereo extraction pipelines. The schema serves three purposes: extracting, encoding, and preserving textual and visual information from the stereograph, bridging stereoscopic collections by mapping their content into a shared structured form rooted in the medium’s intrinsic metadata, and creating the conditions for new data-driven research into the medium itself. The framework is designed to operate on already digitized material, supplementing existing metadata rather than replacing it.

Research Context

Academic discourse around the cataloguing of stereographs remains relatively niche. Goldberg / Clemons (2010) lay a strong foundation through hands-on cataloguing work and a survey of 25 institutional collections, offering practical recommendations based on common pitfalls they encountered. Beyond this, no stereograph-specific metadata schema exists to the best of our knowledge.

With the growing volume of material being archived, AI-generated metadata is increasingly being explored (Oyighan et al. 2024). The application of MLLMs to historical material has been explored for information extraction (Xie et al. 2025), as well as generating metadata and validating existing records (Toth et al. 2025). MLLMs outperform traditional OCR on challenging historical material (Levchenko 2025), and can extract semantic information from visual features, improving search and discovery, and enabling new avenues for research (Arnold / Tilton 2019). However, no published work has applied this approach to stereoscopic photographs to the best of our knowledge.

Methodology

Our proposed schema is an encoding of a stereograph: a structured description complete enough that the JSON could be used to reconstruct the object’s information content. The schema is organized in two layers: a source encoding layer captures observable features of the object (text regions, materiality, visual scene), and a descriptive record layer supports retrieval, cross-referencing, and reconstruction through queryable fields such as title, series, publisher, and date.

The schema is not intended to extend or replace existing standards such as MODS or VRA Core, but to provide a stereograph-specific layer that maps to and complements existing standards while capturing object-level information they were not designed to encode. Records can be populated manually, but the schema is designed for automatic population at scale via an MLLM-based framework.

Our extraction and metadata generation framework uses the Gemini 3 Pro API. Despite being proprietary and expensive, the model was selected for its leading multimodal reasoning, robust multi-language layout-aware OCR, and reliable adherence to complex structured output specifications. The framework consists of four prompts, each populating a different portion of the schema:

  • Text Extraction: performs granular OCR on recto and verso, with strict separation between mount and photographs. Text is clustered into regions based on spatial, visual, and typographical differences. Each region’s textual content, orientation, position, language, legibility, render type (printed/embossed/handwritten), and visual context (caption, series number, publisher, etc.) is recorded in the source encoding layer.
  • Material & Physical Classification: records the stereograph’s physical characteristics, including its physical condition, format (cardboard, glass plate), tonal polarity (positive/negative), curvature, photograph tonality (grey/colour/sepia), frame colour and complexity, stereo pair shape (dome/rectangular) and separation, and the quality of the scan.
  • Visual Scene Captioning: considering the photograph in isolation, the model generates captions and keywords describing the depicted scene. The goal is a frame-independent textual representation of the photograph itself, enabling parity in indexing and analysis between stereographs and conventional photographs.
  • Metadata Synthesis: consolidates the source encoding layer and any existing archive metadata into the descriptive record layer, populating queryable fields such as title, year, series number, publisher, author, and verso text type and content (descriptions, advertisements, etc.), while explicitly recording absences. Existing archive metadata is preferred but verified against the source encoding, with conflicts flagged.

All schema fields that are generated by AI are tagged for transparency. To mitigate hallucinations and ensure consistency, the model is constrained to predefined controlled vocabularies for categorical fields. Outputs are validated against the schema to catch malformed JSON and vocabulary violations, rejecting and regenerating any failures. We evaluate the framework by comparing model outputs against manually populated records to measure OCR and classification accuracy.

Conclusion

Beyond preservation, the proposed schema and framework open a range of further opportunities, particularly when applied at scale. Because the schema encodes information directly from the object in a unified, machine-readable form, the same fields can be compared across collections regardless of how individual archives originally cataloged their holdings. This makes it possible to cross-reference extracted information across institutions to track duplicates, locate surviving copies of fragmented sets, and possibly work toward a master inventory of the medium itself. The same data could also surface broader patterns about the medium itself, for instance the prevalence of piracy (same photo, different frame/publisher), or the influence of specific publishers or photographers. In doing so, it lays the groundwork for a quantitative, data-driven turn in the study of stereoscopic photography, opening new avenues in media archaeology and immersive media history.

The proposed schema is presented as an initial proposition, open to refinement through community feedback and observed deficits.

References
  1. Arnold, Taylor / Lauren Tilton (2019). “Distant viewing: analyzing large visual corpora”. In: Digital Scholarship in the Humanities 34.Supplement 1, pp. i3–i16.
  2. Bethge, Paul Heinrich et al. (2025). “Entangled Gazes: Reconstituting the Stereoscopic Box Set with LLMs and Virtual Reality”. In: Proceedings of the SIGGRAPH Asia 2025 Art Papers. SA Art Papers ’25. Association for Computing Machinery. isbn: 9798400721298. doi: 10.1145/3757369.3767616. url: https://doi.org/10.1145/3757369.3767616.
  3. Darrah, William Culp (1977). The world of stereographs. Gettysburg, PA: W.C. Darrah.
  4. Goldberg, Tyler / Angel Clemons (2010). “Cataloging in Stereo: Metadata for Stereographs”. In: Microform & Digitization Review 39.3, pp. 86–97. doi: 10.1515/mfir.2010.013. url: https://doi.org/10.1515/mfir.2010.013.
  5. Gowda-Storz, Dhruva / Sarah Kenderdine (2025). “Automatic Restoration of Historical Stereoscopic Photographs for 3D Visualization at Scale”. In: Archiving Conference. Society for Imaging Science and Technology, pp. 197–202. doi: 10.2352/issn.2168-3204.2025.22.1.37. url: https://doi.org/10.2352/issn.2168-3204.2025.22.1.37.
  6. Gowda-Storz, Dhruva et al. (2025). “Stereo Spectacular: Reviving the Universal Exposition of 1867 in Virtual Reality through Historical Stereoscopic Photographs”. In: Digital Heritage. Ed. By Stefano Campana et al. The Eurographics Association. isbn: 978-3-03868-277-6. doi: 10.2312/dh.20253121.
  7. Heineman, Joshua (2012). “New Perspectives On Old Perspectives: How An Art Project Helped The NYPL Put Its 3D Stereograph Collection In Your Hands”. url: https://www.huffpost.com/entry/new-perspectives-on-old-p_b_1233351 (visited on 05/07/2026).
  8. Levchenko, Maria A (2025). “Evaluating LLMs for historical document OCR: A methodological framework for digital humanities”. In: Proceedings of the First Workshop on Natural Language Processing and Language Models for Digital Humanities, pp. 75–85.
  9. Luz, Filipe Costa / Rodrigo Peixoto (2015). “The spectacle of stereoscopy: historical analysis and inputs to the creation of a digital archive of analog stereoscopic photography”. In: Proceedings of the IX Congresso Sopcom, pp. 494–504.
  10. Oyighan, Diseiye et al. (2024). “The role of AI in transforming metadata management: Insights on challenges, opportunities, and emerging trends”. In: Asian Journal of Information Science and Technology 14.2, pp. 20–26.
  11. Toth, Gabor Mihaly / Richard Albrecht / Cedric Pruski (2025). “Explainable AI, LLM, and digitized archival cultural heritage: A case study of the Grand Ducal Archive of the Medici”. In: AI & SOCIETY 40.6, pp. 4561–4573.
  12. Vershbow, Ben (2013). “NYPL Labs: Hacking the library”. In: Journal of Library Administration 53.1, pp. 79–96.
  13. Xie, Yunting / Matti La Mela / Fredrik Tell (2025). “Multimodal LLM-assisted information extraction from historical documents: The case of Swedish patent cards (1945-1975) and ChatGPT”. In: The 9th Digital Humanities in the Nordic and Baltic Countries Conference (DHNB 2025), March 5–7, 2025, Tartu, Estonia. University of Oslo Library, pp. 1–15.