DH 2026

Daejeon, July 27–31

Poster

When Data Cannot Move: How to Unlock National Library Collections for Digital Humanities Research at Scale

Marc Kupietz
Leibniz Institute for the German Language (IDS), Germany · kupietz@ids-mannheim.de
Andreas Witt
Leibniz Institute for the German Language (IDS), Germany · witt@ids-mannheim.de
Peter Leinen
German National Library · dnb@leinen-trier.de
Philippe Genêt
German National Library · p.genet@dnb.de
Thora Hagen
Leibniz Institute for the German Language (IDS), Germany · hagen@ids-mannheim.de
Nils Diewald
Leibniz Institute for the German Language (IDS), Germany · diewald@ids-mannheim.de

National libraries have long served as guardians of cultural heritage, systematically collecting and preserving contemporary publications with considerable institutional effort and resources. Increasingly, this mission also encompasses the large-scale acquisition and curation of born-digital and digitized materials. They actively contribute to what we might call the memory of the world in the digital age. Yet these carefully curated collections, which represent the full spectrum of a nation's literary production, have largely remained inaccessible to computational research methods due to copyright constraints (Kamocki 2018, Senseney et al. 2018). The paradox is striking: while libraries invest substantial resources in building comprehensive collections through legal deposit mandates, scholars seeking to study contemporary literature computationally have had to rely on opportunistic samples or commercial datasets that capture only narrow slices of literary production.

This contribution presents DeLiKo@DNB, a project that addresses this disconnect by transforming the German National Library's digital holdings into an accessible research infrastructure, directly advancing the library's mission to open its collections to the Digital Humanities and the broader academic public (Kupietz et al. 2025, 2026). The corpus comprises so far 287,000 contemporary fiction books published digitally in German language between 2012 and 2024, containing approximately 21 billion words. Under the German National Library Act (DNBG) and the Legal Deposit Ordinance (PflAV), publishers are legally required to deposit all digital publications with the library (for similar regulations in other countries, see Roudik et al. 2018). Consequently, this collection represents the entire population of digitally published fiction on the German book market during this period—from literary prize winners to popular genre fiction, and from established publishers to self-published works. Crucially, this statutory mandate means the corpus is not a sample, but the complete landscape of contemporary digital fiction publishing in Germany. It should be noted, however, that the composition of this digital population differs from the population of non-digitally published books, containing, for example, a significantly higher proportion of self-published works (Kupietz et al. 2026). The entire collection is enriched with multiple layers of linguistic annotation (part-of-speech tags, lemmatization, morphosyntactic properties, dependency parsing) and rich metadata. It is freely accessible for non-commercial use at https://korap.dnb.de/.

The project demonstrates how institutional collections can be opened to computational analysis while navigating copyright constraints. Rather than attempting to distribute the data, which is legally prohibited for contemporary copyrighted works, we deploy an already existing, open-source corpus analysis platform (KorAP, Bański et al. 2012) directly at the library, implementing the principle of if the data cannot move, put the computation near the data (Gray 2003). To maximize research utility within the bounds of what is legally permissible, the infrastructure provides a staged access model. At the lowest threshold, researchers can use a web UI without authentication for qualitative and quantitative explorations, such as defining metadata-based virtual corpora, searching for abstract linguistic phenomena using the provided annotation layers, and retrieving concordances. For more complex, reproducible, and programmatic analyses – including diachronic studies, advanced genre comparisons, or custom collocation analyses – researchers can utilize API access supported by client libraries for Rhttps://cran.r-project.org/package=RKorAPClient and Pythonhttps://pypi.org/project/KorAPClient/ (Kupietz et al. 2020, 2022). Across both of these remote access tiers, search results are limited to 50-word snippets, enabling analysis without rights clearance for full-text distribution and without facilitating unauthorized text reconstruction. Finally, at the deepest access tier, researchers requiring unconstrained access to the full texts and metadata can conduct full-scale machine analysis on-site on dedicated devices at the DNB locations in Leipzig and Frankfurt.

For Digital Humanities scholarship, this approach offers several significant advantages. First, it provides unprecedented scale and representativeness of contemporary literature: rather than analyzing convenience samples, researchers can now study the complete documented output of contemporary fiction, enabling questions about literary trends, genre evolution, and linguistic change that require comprehensive data. Second, the rich metadata and flexible virtual subcorpus functionality allow researchers to construct precisely tailored datasets (e.g. to study a specific publisher or genre), with confidence that the underlying collection is exhaustive rather than selective. Third, the multi-layered linguistic annotation framework supports research that combines quantitative discovery with qualitative interpretation, allowing scholars to move seamlessly between distant and close reading. Finally, the project exemplifies a sustainable model for Digital Humanities infrastructure that respects both copyright holders' rights and researchers' needs. The technical pipeline we developed – converting EPUB files to linguistically annotated TEI-XML with automated processing – is fully open source and transferable, offering a blueprint for other national libraries facing similar challenges.https://github.com/KorAP/deliko Initial efforts to enrich metadata through ensemble methods using large language models further demonstrate how modern computational approaches can enhance the discoverability and analytical potential of library collections.

Therefore, this project is also extendable beyond German literature and institutions. The investments libraries make globally in collecting cultural materials need not remain locked away from computational research. By treating libraries as infrastructure providers rather than merely archives, and by designing technical solutions around legal constraints rather than ignoring them, we can unlock data that institutions have painstakingly assembled. As the Digital Humanities increasingly engage with questions requiring data at scale, such collaborations between libraries and researchers become essential.

DeLiKo@DNB illustrates that the resources for Digital Humanities research often already exist within our libraries, awaiting the institutional commitment and technical innovation needed to make them computationally accessible while respecting the rights that protect them. We combine traditional library missions of preservation with contemporary needs for computational analysis, and demonstrate how institutions of memory can become active sites of digital scholarship.

References
  1. Bański, Piotr / Fischer, Peter M. / Frick, Elena / Ketzan, Erik / Kupietz, Marc / Schnober, Carsten / Schonefeld, Oliver / Witt, Andreas (2012): "The new IDS corpus analysis platform: Challenges and prospects", in: Proceedings of the 8th Language Resources and Evaluation Conference (LREC 2012), 2905–2911. https://doi.org/10.63317/4ctnrf49w5hh.
  2. Gray, Jim (2003): Distributed computing economics (Technical Report No. MSR-TR-2003-24). Microsoft Research.
  3. Kamocki, Paweɫ (2018): "The argument for ‘non-consumptive use’ in the EU: how copyright could be redefined to allow text and data mining", in: Intellectual Property Perspectives on the Regulation of New Technologies, 237–258. Edward Elgar Publishing.
  4. Kupietz, Marc / Diewald, Nils / Margaretha, Eliza (2020): "RKorAPClient: An R Package for Accessing the German Reference Corpus DeReKo via KorAP", in: Proceedings of the 12th Language Resources and Evaluation Conference (LREC 2020), 7015–7021. https://aclanthology.org/2020.lrec-1.867/.
  5. Kupietz, Marc / Diewald, Nils / Margaretha, Eliza (2022): "Building paths to corpus data: A multi-level least effort and maximum return approach", in: Fišer, Darja / Witt, Andreas (eds.): CLARIN. The Infrastructure for Language Resources. deGruyter. https://doi.org/10.1515/9783110767377-007.
  6. Kupietz, Marc / Leinen, Peter / Diewald, Nils / Genêt, Philippe / Wilm, Rebecca / Witt, Andreas / Yaddehige, Rameela (2025): "National Library as Corpus: Introducing DeLiKo@DNB – a Large Synchronous German Fiction Corpus", in: Book of Abstracts of DHd 2025. https://doi.org/10.5281/zenodo.14943116.
  7. Kupietz, Marc / Diewald, Nils / Genêt, Philippe / Witt, Andreas (2026): "National Library as Corpus: DeLiKo-2025@DNB – A Very Large Corpus of German-language Contemporary Literature", in: Proceedings of the 15th Language Resources and Evaluation Conference (LREC 2026), 6528–6535. https://doi.org/10.63317/59wsms6588ys.
  8. Roudik, Peter et al. (eds). (2018): Digital legal deposit in selected jurisdictions. Washington, D.C.: The Law Library of Congress, Global Legal Research Directorate. https://www.loc.gov/item/2018299330.
  9. Senseney, Megan / Dickson, Eleanor / Namachchivaya, Beth / Ludäscher, Bertram (2018): "Data mining research with in-copyright and use-limited text datasets: Preliminary findings from a systematic literature review and stakeholder interviews", in: International Journal of Digital Curation, 13(1), 183–194.