Daejeon, July 27–31
Recommender systems (RS) embedded in scholarly search engines influence what researchers see, read, and cite. Yet these systems rarely make explicit the information retrieval (IR) strategies that guide ranking and recommendation. As a result, RS frequently rely on non-semantic metrics such as citation count, reinforcing long-recognized mechanisms of citation concentration also known as the Matthew effect (Merton 1968). Other systems depend heavily on users’ past interactions (Koster/Koch/Kim 2014), creating risks of filter bubbles (Pariser 2011) and confirmation bias in scholarly work (Underwood 2014). With the current shift toward chatbot “AI-assistants” instead of search tools, leaning towards a generic model of queries in natural language parsed by LLMs, questions of alternative modelization, interpretability and epistemic agency are becoming more pressing (Goodlad 2023; Vitali-Rosati 2025). If the mechanisms behind AI-generated recommendations remain opaque, scholars lose the ability to critically assess how relevance is constructed.
This paper asks: what are possible alternative epistemic avenues for “AI” research functionalities? The main contributions of this paper are theoretical, in the shape of an argument towards a DH approach to research tool design that provide an environment that can foster serendipitous findings, and practical in the concrete solution demonstrated through a recommander system designed to answer our research questions as well as the findings of a user study on research and IR practices and the technical evaluation of the LLM-based structured semantic parsing into the Information Economy MetaLanguage (IEML), a language acting as a fixed ontology or a semantic decompositional matrix.
Recent AI-assisted research tools, including LLM-based query expansion, ranking assistants, and automated synthesis, rely on the distributional hypothesis (Firth 1962; Harris 1981), where frequency and co-occurrence patterns act as proxies for semantic relevance. While effective at scale, this approach risks reinforcing existing biases in academic publishing by privileging already-cited literature and narrowing the epistemic space of discovery (de Solla Price 1963; Merton 1968; Nielsen/Andersen 2021; Varga 2022). Evaluations of AI research assistants similarly show persistent opacity, uneven recall, and a tendency to overemphasize mainstream or highly cited material (Patterson et al. 2025; Archambault/Rincón 2024) As Tay (2025) argues, the lack of transparency in contemporary LLM-based IR systems complicates the user’s ability to critically assess results, pointing to the need for approaches that foreground explainability and user agency.
As a goal, serendipity and exploratory search set forth an alternative paradigm in the construction of recommander systems. Seminal work conceptualizes serendipitous discovery as an iterative, user-driven process, “berrypicking” in which queries evolve variants through a non-linear path alternating between document and idea encounters (Bates 1989; Erdelez 1999). Later models highlight the evaluative, user-defined nature of serendipity: a discovery becomes serendipitous only retrospectively, after the user reinterprets its relevance (Lawley/Tompkins 2008; Makri/Blandford 2012). Empirical studies in academic contexts confirm the role of serendipity in cultivating diverse reading practices, fostering cross-disciplinary expertise, and even sustaining research enthusiasm (Foster/Ford 2003). Examples of subverted appropriation of a search-engine’s main functionality suggests that actual research practices are still unmapped and more diverse than current guided practices of IR seem to suggest (Dumas Primbault 2023). Yet most RS remain oriented toward personalization and natural-language querying, aiming for intuitive IR rather than for exploratory breadth or for supporting researchers’ engagement with the complexity of their own specific investigations. A small number of scholarly tools attempt to reintroduce exploration into IR for scientific research. Bridger (Portenoy et al. 2022) exposes researchers to authors outside their usual intellectual networks to promote disciplinary declustering. VITALITY (Narechania et al. 2022) offers a visualization-based approach to literature review but relies on highly curated, pre- processed corpora.
Other projects, such as STAK (Martin/Greenspan/Quan-Haase 2017) have drawn inspiration from the material affordances of physical libraries by attempting to recreate spatial browsing environments such as libraries. Alternatives to LLM-based approaches include knowledge graphs and Semantic Web ontologies. Within symbolic semantic representations, OWL and RDF are well-established formalisms, and the Information Economy MetaLanguage (Lévy 2023) offers a lesser-known but conceptually rich option. IEML defines concepts through nine fixed semantic dimensions (theme/process, who, what, to whom, by what means, when, where, why, how), enabling flexible and heuristic definitions of any concept yet in a structure easy to grasp. Table 1 shows how a concept can be defined within the structure of IEML and semantic connexions made between distant concepts.
While LLMs are increasingly used for semantic parsing tasks, recent evaluations show that they remain unreliable for structured semantic languages. For example, OpenAI models performed poorly on Abstract Meaning Representation (AMR) generation even with few-shot prompting (Ettinger et al. 2023). This reinforces the need for explicit, interpretable semantic frameworks in scholarly IR. In contrast to AMR’s single canonical representation, IEML provides a flexible, user-adjustable semantic decomposition that can support exploratory variation. However, no prior work has examined automatic IEML translation or its integration into recommender systems.
Our main research question is: what kind of alternative to AI research assistant can be designed that can put forward unexpected connections in an explainable and transparent way for the user? To give more perspective to this question: what can a direct comparison between a symbolic ontology and an LLM-based semantic retrieval reveal about their underlying epistemic models? In this work, we suggest that such a tool meant for AI literacy and serendipitous findings should lean on the collaborative and iterative process of query building in an exploratory phase of IR.
To address these questions, we developed a browser-based exploratory recommender system github.com/alexiaschn/ieml-rs (Schneider et al. 2026) integrated with the SSH-focused search engine Isidore. By providing a simple conceptualisation as a grid to be filled (Figure 1), users not familiar with IEML can easily make complex semantic connexions while learning to use the recommander system. If a keyword from the seed article is not present in the IEML database, an automatic translation relying on Gemma-3n-E4B-it is placed on a blank grid (Figure 2).
A side by side comparison of articles queried either from the user-selected concepts and from an LLM-augmented query augmentation provides a powerful visual to the usually invisible impact of LLM-mediation in research practices (Figure 3).
One of the main functionality of the RS is the human-machine collaborative translation into IEML proposed on keywords outside the IEML database as shown in the Figure 2. This functionality relies on a proposed translation done by a machine that the user can correct. These corrected machine-translated keywords are also added to the IEML database following the notion of collective intelligence asserted by IEML’s conceptor (Lévy 1997). For this task, we evaluated three LLMs, Meta-Llama-3-70B-Instruct-Turbo, Gemma-3n-E4B-it and GPT-oss-20B on their ability to translate 360 keywords into IEML using a 60-example few-shot promp and RAG system retrieving 20 selected words from the IEML dictionnary(~3,000 entries). The dataset (including examples given as context) comprises keywords randomly selected from the Isidore search engine. The baseline was the Llama model with no retrieval of dictionary entries.
Quantitative scores (BLEU, cosine similarity) showed negligable gains with contextual examples (RAG) and qualitative analysis confirmed that models fail to reproduce the structural constraints of IEML (Table 2). As noted by a reviewer, BLEU and cosine similarity give only a partial view of the task: vector proximity between chains of concepts concatenated cannot measure accurately the potential relevance of distinct conceptual association however distant they may be from the ground truth, which is where IEML is most valuable. For instance, “francophone” defined in Table 1 uses “to speak” as the root concept, but a valid IEML definition could have put “language” or “French” while omitting speech altogether, which would create a great distance with the ground truth. This confirms findings by Ettinger et al. (2023) on the limits of LLMs in structured semantic parsing, and confirms the importance of humans in the loop in such task. This result hints us towards future designs of recommander systems inclusive of human feedback interaction and interpretations, both cognitive processes fundamental in serendipitous discovery (Makri/Blandford 2012).
After installation and a demonstration of all the features of the plugin, the 6 volunteer users were observed during a 10 minute long session interacting with it, followed by a 15 minute interview covering core functionalities and aspects:
• user-friendliness,
• usefulness,
• keyword navigation,
• translation into IEML,
• article search and panel comparison.
The users were 6 PhD volunteer students in DH and all knew the developper. The authors recognize that both elements account for important biases in this study and limits the scope of the following analysis.
Observations from the experiment indicate that all participants extracted articles through the RS, suggesting that the tool facilitated access to previously unknown literature even for those who do not typically rely on keyword search. Although all users formulated multiple requests and compared article lists to varying extents, only one engaged equally with both stages. All but one corrected LLM-generated translations, and one participant annotated rather than replaced them, indicating that low-quality outputs may prompt a dialogic, collaborative stance toward semantic tasks. Such imperfect translations appear to foster reflexivity in the articulation of semantically rich concepts and support the user’s appropriation of the tool. Given the limited number of pre-translated keywords (420 at the time of writing), users frequently encountered “untranslated” concepts; rather than a simple constraint, this gap constitutes an opportunity to enhance user agency by inviting them to formulate the missing conceptual mappings into IEML and participates in the appropriation of the tool itself.
Participants recommended various feature enhancements and UX adjustments, yet many limitations appear solvable through direct integration within the host search engine (e.g. maintaining session state, providing full metadata, distinguishing more clearly between query construction and document retrieval, and mitigating latency caused by chained API calls).
To the best of our knowledge, this tool provides the first request-building application involving semantic conceptualisation performed collaboratively by a human and a LLM. As a first use case for the automatic semantic parsing into IEML, the tool confirms the current limitations of the now-common architecture RAG for word association and structured definition. The primary contribution of this article lies in its argument that recommender systems function as critical instruments for revealing, rather than concealing, divergences in upstream information retrieval modelization —specifically between ontology-based approaches and the distributional hypothesis. Furthermore, the paper advocates for the strategic value of "friction" as an intentional design principle in research-oriented human-computer applications.