Annotating is one of the seven “scholarly primitives” proposed by John Unsworth, one of the “basic functions common to scholarly activity across disciplines, over time, and independent of theoretical orientation” (Unsworth 2000). Annotation—traditionally of text but also of images and, more recently, of audio and video materials—is an essential scholarly practice in the humanities, in digital humanities (DH), and beyond. As information is increasingly created, studied, and disseminated in a digital environment, digital tools for practices like annotation must not only be developed, but also made accessible and findable for students and researchers alike. As the University of Alberta’s Annotator in Residence, a position I have filled since January 2025, I have been researching audiovisual (AV) annotation practices, collecting and developing representative examples of annotated AV materials, and working towards creating an AV Annotation Use Case Recipe Book. AV annotations can take many forms, including timestamped indexes, transcripts, translations, citations, comparisons between two or more AV files, close listening notes, and many other forms of documentation and analysis. Inspired in part by The Cookbook of IIIF Recipes and The Programming Historian’s tutorials, this Recipe Book is a structured and searchable collection of use cases for AV annotation tools, such as AVAnnotate.

Created by Dr. Tanya Clement at the University of Texas at Austin, AVAnnotate is a minimal computing, open-source application and workflow used for creating and sharing annotations of audio and video materials, as well as for creating digital exhibitions and editions with these materials. Clement and Wintermeier describe annotation as an affordance of AVAnnotate, where annotations of AV materials “afford opportunities for particular communicative and symbolic actions in relation to AV artifacts” (Clement / Wintermeier 2025: 1), including close listening and the articulation of the spatial, aural, and temporal elements of the artifacts.

Annotation can thus be a powerful tool for critically engaging with audio and video materials. While the annotation of AV materials includes well-established practices such as transcription and translation, AV annotation remains a comparatively recent development in contrast to the extremely long history of text annotation. The practical use of AV annotation tools can be limited not only by a lack of understanding about how to use these tools, but also by a lack of awareness about the circumstances in which these tools are useful. The Recipe Book aims to provide students, instructors, and researchers with clear, simple, and brief use cases for AV annotation tools, including user stories, brief descriptions, step by step instructions, and links to working examples, both to educate and to stimulate interest and engagement with the practice of AV annotation. While the Recipe Book project originated with AVAnnotate, annotating AV materials extends beyond a single tool. As Pacheco (2022) concluded in an analysis of the ontology of DH, digital tools in DH serve to amplify the research being done—the priority is the utility of the tools and the needs of scholarly research, rather than the features (and limits) of the tools themselves. Thus, the Recipe Book explores AV annotation in general, with AVAnnotate presented as one example of a digital annotation tool.

As well as presenting the AV Annotation Use Case Recipe Book itself, this paper will also outline the process of creating the Recipe Book, including the collection and development of use cases, and the development of a minimal computing website to host the Recipe Book. Rather than building the Recipe Book directly using a tool like AVAnnotate, open-source static site generators like Hugo and Jekyll were explored as options to develop the website. This allows for the Recipe Book to be flexible and not tool-dependent, while embracing the minimal computing ethics espoused by AVAnnotate (Clement et al. 2022; Clement / Wintermeier 2025). The flexibility of Hugo and Jekyll also allows for features to be added to the Recipe Book to improve its usability and searchability, such as the inclusion of a search function as well as tags and filters, allowing for use cases to be searched by factors such as purpose (teaching, research, presentation, etc.), type of AV content (podcast, radio broadcast, documentary video, etc.), and by relevant scholarly field. The Recipe Book is not limited to humanities-based use cases, and includes use cases such as highlighting data points by annotating marine biology diving videos and wildlife research field recordings. While creating and collecting examples of AV annotation projects outside of the humanities has been a challenge, there are plans to expand the Recipe Book in the future by soliciting additional use case “recipes” from researchers across disciplines. Rather than a static resource, the Recipe Book aims to be a living, cross-discipline, collaborative project which is both useful to and built by a diverse group of scholars engaged with the annotation of AV materials.