Daejeon, July 27–31
Nineteenth-century British newspapers were permeated by uncertainty. A correspondent might write: “A rumour is current that Garibaldi is dead” or “We hear a rumour of war between France and Austria.” Such sentences were how publics first encountered — and contested — consequential events. PRESTO (Pattern-based Rumour Extraction with Semantic Tracking Over historical newspapers) is a two-phase methodology for discovering rumour topics in historical newspapers and tracking how they are discussed across time, outlets, and narrative contexts in a large OCR’d corpus. Computational approaches typically either cast a wide net with keyword search, producing overwhelming noise, or apply classifiers difficult to validate on OCR-degraded text. PRESTO treats rumour as a structured linguistic act: the keyword ‘rumour’ and its variants serve as a high-precision entry point for locating candidate sentences, but the output of Phase 1 is not a keyword-matched document list — it is a set of structured propositional claims extracted via dependency parsing, a materially richer representation.
This paper addresses four connected questions:
Digitised historical newspapers pose well-known challenges—OCR noise, heterogeneity, and collection biases—motivating transparent, reproducible workflows in DH (Ehrmann et al., 2023; Beelen et al., 2023). Event-oriented access has been explored via multilingual unsupervised event detection (Boros et al., 2022). In NLP, uncertainty has been operationalised through hedge detection and event factuality annotation (Farkas et al., 2010; Sauerí / Pustejovsky, 2009). More recent work includes the SemEval RumourEval shared tasks (Gorrell et al., 2019) and transformer-based veracity classification (Kochkina et al., 2023), which demonstrate the progress achievable when training data is plentiful and text is clean. PRESTO is complementary: rather than classifying veracity, it reconstructs historical discourse at the claim level from structurally degraded archival text where supervised classification is impractical. PRESTO uniquely treats rumour framing as a structured linguistic act — recovering the claimed proposition rather than assigning a veracity label — opening new pathways for humanities-oriented analysis of how uncertain information circulated and transformed across the Victorian press.
We work with the British Library’s Heritage Made Digital newspapers distributed as the biglam/hmd_newspapers dataset (3,065,408 articles, CC0-1.0). The corpus contains OCR’d articles with metadata and quality indicators. PRESTO is designed to be reproducible: all steps use open-source tooling with explicit thresholds so the pipeline can be rerun on other corpora or periods. The PRESTO Corpus (7,460 structured rumour claims) and all accompanying notebooks are openly available at https://doi.org/10.5281/zenodo.20077470 under CC0-1.0. Each entry includes the original matched sentence, extracted rumour proposition, pattern type, named entity annotations (PERSON, GPE, ORG, etc.), publication metadata, and OCR quality scores.
PRESTO employs a nine-step pipeline. Article-level filtering applies r’\brumou?r(?:s|ed|ing)?(?:'s)?\b’ to identify 78,207 candidate articles:
PRESTO employs a nine-step pipeline. Article-level filtering applies r’\brumou?r(?:s|ed|ing)?(?:'s)?\b’ to identify 78,207 candidate articles:
This yields 7,460 claims averaging 15 words. See Figure 1 for the complete workflow.
Figure 1: Workflow for OCR historical newspaper text rumour content extraction and articles collection
Phase 1 drops about 90% of initial candidates. This is the cost of structure: the average retained sentence is about 175 characters (30 words) with multiple subordinate clauses; 32.8% of extractions use that-clauses and 26.5% use of-patterns. Dependency parsing for such long chains is effectively binary — OCR noise corrupting a single bridge token or named entity causes extraction to fail even when the sentence remains humanly readable. We interpret the 90% reduction as structural noise rather than topical irrelevance, and make this precision/coverage trade-off explicit in our analysis and evaluation.
Phase 2 starts from a Phase 1 claim and retrieves broader discourse including reporting that omits rumour framing (documented in the accompanying notebooks). From a seed expression we generate paraphrases by combining WordNet-derived (Fellbaum, 1998) morphological and derivational variants with controlled templates for syntactic alternation and tense variation, including perfect tenses such as has died and had died. For example, from ‘war between France and Austria’ the generator produces: Franco-Austrian war, French-Austrian conflict, Austro-French hostilities, and the war between France and Austria has broken out. Candidates are ranked by cosine similarity using sentence-transformer embeddings (all-mpnet-base-v2; Reimers / Gurevych, 2019; https://huggingface.co/sentence-transformers/all-mpnet-base-v2) and the top-scoring expressions drive corpus-wide retrieval, followed by sentence- and article-level deduplication.
To avoid overclaiming recall, we use pooled relevance judgments: results from three retrieval strategies are merged into a single set and labelled by a human adjudicator, allowing systematic comparison without exhaustive annotation of millions of articles. The three strategies are strict exact-phrase matching (high-precision baseline), OR-over-variants using PRESTO’s generated expressions, and a BM25 lexical baseline (a probabilistic term-frequency ranking function; Robertson / Zaragoza, 2009) built over the pre-filtered sentence set with a multi-term query of the canonical phrase and its top variants. We report Precision@k and Recall@pool, where Recall@pool measures coverage relative to the jointly labelled set only — not absolute recall over the full archive — which is the methodologically honest claim available without exhaustive annotation. Items are labelled as relevant when they clearly assert, report, or discuss the target event, including reported-but-denied claims; when a second annotator is unavailable, an LLM flags disagreements for triage and a human adjudicator assigns all final labels. When a second annotator is unavailable, an LLM is used only for triage; all final labels are assigned by a human adjudicator.
Across three events: for the death of Garibaldi (pool n = 23, relevant = 23), exact-phrase matching achieves P@10 = 1.00 but Recall@pool = 0.35 (8 of 23); OR-over-variants reaches Recall@pool = 0.48 (P@10 = 0.90); BM25 reaches Recall@pool = 0.83 (P@10 = 0.10). For the France–Austria war (n = 733, relevant = 725): exact-phrase Recall@pool = 0.13, OR-over-variants = 0.55, BM25 = 0.54. For the Russia–Turkey war (n = 467, relevant = 467): exact-phrase = 0.67, variants = 0.78, BM25 = 0.86. PRESTO’s expansion step consistently increases coverage over strict canonical phrasing while Phase 1 provides interpretable structured seeds. The quadrupling of recall in the France–Austria case study — from Recall@pool = 0.13 with exact-phrase matching to 0.55 with OR-over-variants — demonstrates that a single canonical phrase captures only a fraction of how a rumoured event was expressed across titles and decades, and that semantically ranked expansion is necessary for reliable longitudinal tracking.
PRESTO contributes a reproducible method for moving from explicit rumour frames to event-scale discourse tracking in historical newspapers, combining pattern-constrained dependency parsing with similarity-ranked query expansion. The PRESTO Corpus is a silver-standard dataset of 7,460 automatically extracted rumour claims; manual annotation of a random 200-entry sample yielded extraction precision of 67.0% (95% CI: 60.2–73.1%), with errors attributable to OCR-damaged parse bridges (29%), inadvertent capture of rumour meta-framing (21%), and span truncation (20%). The corpus and annotated validation set are available at https://doi.org/10.5281/zenodo.20077470 (CC0-1.0). Phase 2’s semantic expansion quadrupled recall in the France–Austria case study over strict canonical phrasing. Limitations: Phase 1 is inherently conservative; the method relies on English-language resources (NLTK, Stanza, WordNet), limiting portability; and full data-loss measurement requires manual annotation of dropped sentences at scale, which remains future work; the 67.0% precision estimate confirms the corpus is suited for exploratory and retrieval-oriented analysis rather than direct quantitative use without further verification. This foundation can be extended to track propaganda, hoax, and gossip, and we treat PRESTO as a transparent, precision-oriented basis for mixed-method DH work rather than a complete solution to OCR variability.
This technical foundation can be immediately expanded to track related concepts such as propaganda, hoax, and gossip, enabling broader historical and sociological analysis of information uncertainty across the press. Beyond the British context, the same pipeline architecture is applicable to any OCR’d newspaper corpus with sufficient NLP infrastructure, making PRESTO a reusable blueprint for cross-national comparative studies of how unverified claims circulated before the era of mass communication standardisation.