DH 2026

Daejeon, July 27–31

Thu, July 3009:00–10:30S058106
Short Paper

When Systems Do Not Decide: Litigation, Governance Documents, and Meaningful Human Control in Canadian Immigration AI

Ralph Padilla
University of Alberta, Canada; Bridging Divides U of A · ralphiva@ualberta.ca
Geoffrey Rockwell
University of Alberta, Canada; Bridging Divides U of A · grockwel@ualberta.ca
Yasmeen Abu-Laban
University of Alberta, Canada; Bridging Divides U of A · yasmeen@ualberta.ca

Across civil services, artificial intelligence and advanced analytics systems are increasingly deployed as decision support infrastructures rather than automated decision-makers. These systems are typically framed as neutral tools that enhance efficiency while preserving meaningful human control, with state officials retaining final authority over outcomes. This paper critically examines that framing through a close reading of the Integrity Trends Analysis Tool (ITAT), an advanced analytics system developed by Immigration, Refugees and Citizenship Canada (IRCC). Known successively as Watchtower, Lighthouse, and now ITAT, the system's naming history is itself analytically significant: each iteration moves further from the metaphor of surveillance toward the language of neutral technical infrastructure, a shift that mirrors broader discursive strategies for governing algorithmic accountability (Amoore 2023; Chartier-Edwards et al. 2025).

The paper draws on two categories of primary sources whose different provenances are themselves analytically significant. The first is material published voluntarily by IRCC via Canada's Open Government Portal, principally the Algorithmic Impact Assessment (AIA, updated March 2025). The second, and more extensive, is a corpus of internal governance documents that entered the public record not through institutional transparency initiatives but through litigation: the affidavit of Wei William Tao, submitted in Fatemah Mehrara et al. v. The Minister of Citizenship and Immigration (IMM-6463-23, Federal Court of Canada, 2024), a document exceeding one thousand pages. This affidavit includes Director General Steering Committee intake decks (July and August 2022), an Executive Committee launch approval document (December 2022), a Model Privacy Assessment (May 2021), a Gender-Based Analysis Plus report, internal correspondence, and a peer review of the system, then called Watchtower, conducted by Statistics Canada's Data Science Division and Data Ethics Secretariat. The contrast between what IRCC chose to make public and what litigation compelled into visibility is not incidental to this paper's argument: it is a central instance of the accountability dynamics the paper seeks to analyze. This approach draws on digital humanities traditions of close reading and critical document analysis, examining how institutional texts participate in meaning-making even when framed as administrative formality (Gitelman 2013; Drucker 2011).

ITAT is described by IRCC as a data-mining tool that identifies risk patterns in immigration applications and flags cases for verification by Risk Assessment Units. Official accounts consistently emphasize that the system does not automate decisions, does not directly determine approval or refusal, and does not replace officer discretion. Yet the documents reveal a more complex picture. The "Before and After" workflow diagrams circulated at the Director General and Executive Committee levels show that ITAT's introduction restructures the processing pipeline in a way that relocates bias screening upstream and outside officer awareness. Specifically, in the future-state workflow, applications are assessed against ITAT patterns at the moment of receipt, before any officer engagement, and the results of this assessment are withheld from processing officers entirely. Officers see only the downstream outcomes of verification activities initiated, with or without ITAT involvement, by RAUs. The system is, as the AIA acknowledges, "two steps removed" from the final decision. But it is also, by design, prior to and invisible within the steps that follow.

The AIA itself produces a revealing set of internal contradictions. The algorithm is declared a trade secret, yet the process is simultaneously described as not difficult to interpret or explain. The Gender-Based Analysis Plus holds that the system treats applicants from different countries, genders, and age groups in a manner "consistent with the historical risk distribution," a formulation that encodes historical patterns of differential scrutiny as a baseline rather than a problem of possible bias to be corrected. The peer review conducted by Statistics Canada's Data Ethics Secretariat identifies these tensions directly: it challenges IRCC's claim that the system is "not predictive" as inconsistent with the system's own operational logic, asks whether "adverse information" functions as a euphemism, raises the question of whether applicants could be repeatedly investigated following non-adverse findings, and recommends the publication of documents that were, at the time, not yet public. These contradictions are not incidental. Rather, they reflect a governance architecture designed to limit the scope of accountability to formal decision points while leaving upstream infrastructural processes less visible (Chartier-Edwards et al. 2025; Fourcade / Gordon 2020).

This paper argues that focusing on whether ITAT makes decisions obscures its broader governance significance. In particular, the more consequential question is how it reshapes the epistemic and temporal conditions under which decisions are made: which cases receive enhanced scrutiny, at what moment in the processing pipeline, and through what institutional channels. ITAT's pattern reports function as a form of machine-generated annotation, translating complex administrative histories into actionable risk signals before any officer encounters an application (Drucker 2011; Gitelman 2013). These reports are framed as factual and self-contained, yet they perform interpretive work by foregrounding certain associations, encoding historical risk distributions as current baselines, and rendering the provenance of that framing invisible to the officers whose judgment it seems poised to precondition.

Drawing on philosophical accounts of meaningful human control, we hold that control requires not only the formal presence of a human decision-maker, but the substantive capacity to understand, trace, and intervene in the conditions that structure judgment (Santoni de Sio / van den Hoven 2018; Robbins 2023). When officers exercise discretion without awareness that an application has been flagged, or without access to the pattern that prompted verification, the conditions for meaningful control are structurally compromised, not through any discrete failure, but through the routine, distributed, infrastructural operation of the system as designed (Cornelissen et al. 2022; Himmelreich / Kohler 2022; Davidovic 2023).

The contribution of this paper is threefold. First, it extends digital humanities approaches to critical document analysis into the domain of administrative AI governance, treating governance documents as interpretive artifacts rather than neutral records, and demonstrating how the provenance of those documents, whether voluntarily disclosed or compelled through litigation, is itself a form of evidence about institutional accountability. Second, it demonstrates how internal contradictions across a multi-document corpus, between the AIA, the GBA Plus, the peer review, and the workflow diagrams, reveal accountability gaps that no single document would expose. Third, it deepens discussions of meaningful human control by shifting focus from formal decision authority to the infrastructural conditions upstream of that authority, conditions that this corpus of documents makes legible for the first time as a coherent analytical object.

Keywords: meaningful human control; algorithmic governance; critical document analysis; immigration; infrastructural steering; accountability.

References
  1. Amoore, Louise (2023): "Machine Learning Political Orders", in: Review of International Studies 49, 1: 20–36.
  2. Chartier-Edwards, Nicolas / Blottiere, Mathieu / Roberge, Jonathan (2025): "AI Statecraft Heating-Up: The Automation of Governance through Canada's Chinook Case Study", in: AI and Society 40: 765–774.
  3. Cornelissen, Noel A. J. / van Eerdt, Ramon J. M. / Schraffenberger, Hanna K. / Haselager, Wijnand F. G. (2022): "Reflection Machines: Increasing Meaningful Human Control over Decision Support Systems", in: Ethics and Information Technology 24: Article 19.
  4. Davidovic, Jovana (2023): "On the Purpose of Meaningful Human Control of AI", in: Frontiers in Big Data 5: Article 1017677.
  5. Drucker, Johanna (2011): "Humanities Approaches to Graphical Display", in: Digital Humanities Quarterly 5, 1.
  6. Fourcade, Marion / Gordon, Jeff (2020): "Learning like a State: Statecraft in the Digital Age", in: Journal of Law and Political Economy 1, 1: 78–108.
  7. Gitelman, Lisa (ed.) (2013): "Raw Data" Is an Oxymoron. Cambridge, MA: MIT Press.
  8. Himmelreich, Johannes / Kohler, Susanne (2022): "Responsible AI through Conceptual Engineering", in: Philosophy and Technology 35: Article 60.
  9. Robbins, Scott (2023): "The Many Meanings of Meaningful Human Control", in: AI and Ethics 4: 1377–1388.
  10. Santoni de Sio, Filippo / van den Hoven, Jeroen (2018): "Meaningful Human Control over Autonomous Systems: A Philosophical Account", in: Frontiers in Robotics and AI 5: Article 15.
  11. Tao, Wei William (2024): Affidavit of Wei William Tao in Fatemah Mehrara et al. v. The Minister of Citizenship and Immigration, IMM-6463-23. Federal Court of Canada.