Daejeon, July 27–31
For Harvard University’s Arts and Humanities Research Computing unit (DARTH), the increased instability of digital humanities funding threatens both new and existing DH projects, especially those that take the form of dynamic, database-driven research tools. Built for faculty researchers to perform rich queries across complex data models, these projects are expensive to build as well as maintain. Nonetheless, faculty continue to view these sophisticated, customized discovery tools as indispensable to their research. This presentation introduces DARTH’s use of the free, client-side static search library Pagefind to balance search, sustainability, and budget in DH projects. Three case studies—Tsumeb Mine Notebook, the Amendments Project, and the Black Teacher Archive—illustrate how Pagefind can be used to build new projects, sunset legacy databases, and embed preservation planning from the start, all with greatly reduced financial and engineering resources.
In recent years, many DH practitioners have appropriately shifted toward Static Site Generators (SSGs) to address pressing sustainability concerns. This “minimal computing” approach simplifies the overall technological stack, reduces maintenance overhead substantially, and enables project hosting on highly inexpensive or even free services. While static preservation effectively guarantees the long-term availability of project files, this strategy introduces a significant trade-off: it often sacrifices the sophisticated search interfaces necessary for scholarly interpretation and analysis. When a dynamic site is sunsetted to a static form, the underlying database may be archived on specialized platforms, such as Dataverse. https://dataverse.harvard.edu/.
Humanities faculty at Harvard cite sophisticated, customized discovery tools as key to their research. Many faculty-led projects involve meticulously collected and cleaned data, typically organized, added, and enhanced within PostgreSQL databases which then serve dynamic front-end websites. https://www.postgresql.org/. https://www.elastic.co/elasticsearch/.
Declining funding has put further pressure on this functionality gap between the needs of researchers and the limitations of sustainable technology. DARTH is directly addressing this challenge by using Pagefind, a free, open-source, client-side static search library released in 2022. https://pagefind.app/.
Pagefind’s unique indexing process makes it superior to existing static search tools, especially for large datasets or complex research requirements. Typically, Pagefind begins by locating and parsing the HTML files within the static site directory, extracting textual content and organizing it into searchable blocks. Each block is treated as an identifiable and individual segment. During the tokenization process, every token is mapped back to the specific block and page where it appears. Crucially for performance, Pagefind then "chunks" the resulting index into small pieces, which are compressed to facilitate faster retrieval by the browser. When a user initiates a search on the site, the browser is only required to load the necessary pieces of the index. By comparison, libraries like LunrJs and FlexSearch.Js require the user to download the entire search index, slowing down page load speed and decreasing performance. https://lunrjs.com/; https://github.com/nextapps-de/flexsearch.
To demonstrate Pagefind’s practical utility and its role in meeting the challenges of search, sustainability, and budget, the presentation will offer three distinct case studies, each representing a different stage in a DH project’s lifespan: planning a new project, sunsetting a legacy project, and designing a project for static maintenance. DARTH’s implementation of Pagefind across these projects utilizes the Pagefind JavaScript API via a set of reusable Vue components created internally. https://vuejs.org/
The first case study focuses on a new project, the Tsumeb Mine Notebook. https://tmn.fas.harvard.edu/. https://astro.build/; https://docs.github.com/en/pages.
The second case study, the Amendments Project, illustrates Pagefind’s role in responsibly solving the pervasive problem of "zombie" projects often left stranded on university servers. https://amendmentsproject.org/. https://www.djangoproject.com/.
The final case study involves the Black Teacher Archive, focusing on the planning phase of a new project. https://bta.gse.harvard.edu/. https://svelte.dev/
For DARTH, Pagefind has become a proven means of preserving the analytical richness inherent in DH projects while simultaneously achieving high sustainability with reduced resources. It is not meant as a blanket solution, but rather a new strategy for pursuing a balance across user engagement, longevity, and budget. As the field of digital humanities adapts to new technological and financial circumstances, we hope this presentation will contribute to a broader discussion about the import and viability of custom, public-facing research tools in future humanities scholarship.