Daejeon, July 27–31
From the 1910s through the 1980s, Hollywood studios promoted their movies through the creation and dissemination of pressbooks—bound pamphlets containing publicity materials, advertising layouts, accessories for sale, and other promotional tactics (Hoyt, 2022). These promotional booklets were sent to exhibitors and press outlets, making them vital nodes within the wider networks of film circulation and culture. Notably, many pressbooks contained pre-written newspaper articles, intended for theater managers to place in their local newspapers. However, film and media studies scholars have yet to explore the reach of these publications: who used them, how, and whether the publicity text, promotional photos, and ads from the pressbooks permeated American newspapers and magazines as intended.
Accordingly, Project Ballyhoo investigates the reach of these publications by running similarity analyses between Hollywood pressbooks and Chronicling America, an expansive collection of local newspapers from throughout the United States (Library of Congress, 2025). High-throughput computing infrastructure and sophisticated machine vision workflows enable us to examine these corpora at scale to search for text similarity and reuse (Hoyt et al., 2025). We used computer vision to identify and classify separate articles, images, and other elements from each pressbook page before running optical character-recognition (OCR) on individual elements. This data workflow enables us to directly compare the Hollywood pressbooks to more than 1 billion elements identified by Dell et al. in the Chronicling America collection (Dell et al., 2023). We intend to report our specific results in other venues. In this paper, therefore, we grapple with issues of labor equity, questions of intellectual contribution, and interrogate the nature of knowledge production within the context of machine learning.
We are concerned that many critical contributions to computer vision projects may be overlooked unless issues of authorship and contributorship are directly confronted and discussed throughout the project’s stages. In this paper, we argue that the work done by data annotators is intellectual labor that warrants contributorship attribution and may meet criteria for authorship in some circumstances. The proliferation of AI and AI-adjacent technologies necessitate re-examinations of authorship practices across the digital humanities. Therefore, we conclude with recommendations for principal investigators, journals, and other institutions to confront these issues
Text similarity and reuse detection methods have frequently appeared in digital humanities (DH) projects throughout the field’s development (Ladd 2020). The ability to rapidly compare enormous volumes of text at scale is useful across disciplines to answer questions that would not be possible with human labor alone. However, projects that have conducted such text comparisons tend to run against the practical limitations of computing power (Backurs and Indyk 2014) and require bespoke solutions for specific datasets to enable parallel processing of data (e.g., Bao et al. 2016; Svyatkovskiy et al. 2016; Smith, Cordell, and Mullen 2015; Chitturi 2021). While approach increases the complexity of work, many DH projects have successfully adopted this technique.
One influential example is Ryan Cordell and David Smith’s “Viral Texts” project, which tracks the circulation of poetry and other short-form literature in 19th-century US newspapers (Cordell and Smith, 2022). Other exemplary projects include Lincoln Mullen’s “America’s Public Bible”, which tracks the use of Bible passages in US newspapers, showing how Christianity contributed to the “imagined community” of the United States (Anderson, 2016; Mullen, 2022). Large-scale text comparison has also been applied beyond newspaper collections. The EarlyPrint project, for example, created a linguistically annotated and deeply searchable archive of books and broadsides printed between 1473 and 1700 (Mueller and Loewenstein, 2018). Additionally, Kellen Funk and Lincoln Mullen have applied text similarity and reuse detection techniques to examine the recirculation of specific phrases in legal documents (Funk and Mullen, 2018).
These examples demonstrate the utility of text similarity methods in DH projects. Image-based comparisons, however, have been less common among DH practitioners. This is unsurprising, given that the work is more computationally complex. Furthermore, visual content often employs nuanced representational strategies which may be experienced by a human viewer as affect, but which elude clear articulation or definition (Barthes, 1977; Hall, 1980; Scott, 1999; Sontag, 2010). In their description of “distant viewing” methods, Taylor Arnold and Lauren Tilton explain that, “while a person looking at an image can decode the objects and meaning of the visual messages, the process of making these explicit decisions is what makes images so challenging to study computationally” (Arnold and Tilton, 2023: 15).
Many DH projects have relied on the collective efforts of many to analyze culture at scale. Indeed, numerous projects have demonstrated the efficacy and practicality of using crowdsourced labor for image annotation tasks (Downs et al., 2010; Hipp et al., 2013; Jofre et al., 2020; Kittur et al., 2008; Nowak and Rüger, 2010). However, there may be an unspoken assumption that outsourcing or crowdsourcing work such as data annotation may leave the researchers free to focus their attention on other (and seemingly more significant) components. John Unsworth describes annotation as a “scholarly primitive,” that is, it “[forms] the basis for higher-level scholarly projects” (Unsworth, 2000). We reject this framing of “higher-level” work (and its implied inverse) and instead call on DH scholars and ML practitioners to recognize the value of data annotation as intellectual labor. Others, such as De Weerdt et al. also advocate for a more expansive thinking. They assert that “the design of annotation standards, structures, and topologies are also always acts of interpretation” (De Weerdt et al., 2025, para. 14). Our work on Project Ballyhoo affirms this perspective and we urge DH scholars and practitioners to take seriously the intellectual work of dataset annotation.
In their critical analysis of the popular ImageNet dataset, Denton et al. caution that “comparatively little attention has been paid to the various modes of their constitution” (Denton et al., 2021: 1-2). This oversight is troubling, particularly given the numerous issues of equity associated with crowdsourced labor. One study found that crowdsource workers tend to be young, female, and low-income workers from India and the United States (Casey et al. 2017). In a critique of Amazon Mechanical Turk, Lilly Irani notes that “These workers absorb the tedious work necessary but unbecoming of idealized “creative” workers” (Irani, 2015: 729). We are wary of a similar hierarchy emerging in the context of intellectual labor.
For Sara Ahmed, “Citation is how we acknowledge our debt to those who came before; those who helped us find our way when the way was obscured because we deviated from the paths we were told to follow” (Ahmed, 2017: 15). There are many structural inequities pervasive to academia, and our call to action is on possible corrective (Khodakarami et al., 2025; Larivière et al., 2016; Ni et al., 2021). We follow the Citational Justice Collective’s view of the politics surrounding knowledge production. They call on scholars to re-examine their assumptions surrounding authorship, suggesting possibilities for broader perspectives: “Motivating a paper, organizing collaborative work, discussing findings, and doing data collection could be explicitly recognized” (Citational Justice Collective et al., 2022: 81). Such an analytical move is crucial; In 2019, Brittnay Edmonds summarized the state of academia like this: “institutions continue to consolidate and underfund departments historically engaged with the study of difference, demote departments to programs, make precarious and overburdened staff and faculty, and increasingly work to reproduce the university as a site of class privilege and exclusion” (Edmonds, 2019: 75). In the following years, such attacks on academia have only accelerated and in the United States, dramatic cuts to federal research funding have directly affected our work. This tumultuous context reminds us of the significant ethical stakes of all our research.
In 2025, a team of three graduate students (FitzWhittemore, Proctor, and Riley) spent two months identifying and classifying image segments from 3,832 page scans. Hansen augmented this annotation data using standard techniques (shearing images, rotating bounding boxes, cropping, etc.) and trained a YOLOv11 image segmentation model (Khanam and Hussain, 2024). The original images, annotation data, and model weights are all publicly available in an open-source data repository for other researchers to build upon (Proctor et al., 2025).
The full team held meetings throughout the annotation process to check-in and share findings from the annotation process. Data annotators also used a Slack channel to collaborate and discuss images that were difficult to classify. After completing the image annotations, Pettis facilitated a discussion with the annotators to reflect on the overall process. We understand these after-the-fact conversations as a necessary form of critical reflexivity that acknowledges the labor of data annotators while also attending to the myriad ways that the particularities of the annotation process unavoidably shape the output model(s) and other downstream research findings. This DH2026 presentation emerges from these ongoing conversations and our understanding of the intellectual work of data annotation which coalesced.
Data annotation is a time-consuming and demanding task. Even working at a steady pace, the total process took nearly two months to complete. And while the work is repetitive and tedious, the annotators all recognized how simple decisions –for instance, whether text like “Turn to the next page for an exciting layout…” should be classified as a caption or a headline–could have important implications for the eventual computer vision model’s accuracy. As FitzWhittemore put it, “the data annotator is constantly making microdecisions about how content should be understood and classified.”
Ignoring the intellectual contributions of data annotators is a form of “ghost authorship,” and continually overlooking this labor can be detrimental to research (McNutt et al., 2018). Accordingly, we offer the following recommendations for projects that rely on large-scale data annotation.
We recognize that a “one-size-fits-all” approach to authorship and contributorship is unlikely work for all data annotation projects. However, given the well-documented inequities which exist throughout academia, it is imperative that DH practitioners take seriously the intellectual labor of data annotation at all stages of their projects.