Daejeon, July 27–31
Recent advances in large language models (LLMs) have improved local coherence and stylistic fluency; however, these systems continue to struggle with the global structures required for long-form fiction (Goldfarb-Tarrant et al. 2025). When left without guidance, LLMs tend to follow statistically dominant continuations, producing narrative drift, weakened causality, and reduced thematic range (Shi et al. 2025). These limitations highlight a central challenge in computational text generation: next-token prediction alone cannot sustain the structural variation that characterizes literary narrative (Teleki et al. 2025). Research in narratology and computational creativity has shown that mid-level structures shape creative outcomes without prescribing specific narratives, aligning with accounts that describe formal structures as shaping pacing and interpretation (Tenen 2017) and with analyses that treat infrastructural organizations as conditions of generative possibility rather than limitations (Tenen 2024). Importantly, as this line of scholarship demonstrates, early twentieth-century literary formats already embodied proto-computational logics that balanced stability and variation. The Plot Genie (Wycliffe 1931) can thus be understood not as a historical curiosity but as an early narrative algorithm designed to regulate creative possibility.
Motivated by this insight, we propose Craft-in-the-Loop (CITL), a multi-agent system for long-form novel generation by integrating the indexed schemas of The Plot Genie with Chatman’s theoretical framework (1975). Comparative evaluation against human-written novels and a Vanilla-GPT baseline demonstrates that moderately calibrated narrative constraints significantly enhance creative diversity and narrative originality.
Narratology and computational creativity offer complimentary frameworks for explaining why structure is essential in generative systems. Vauth et al. (2021) argue that narrativity depends on multi-scalar organization, while Gius and Vauth (2022) show that tellability arises from inflection points rather than lexical fluency. Empirical evaluations of long-form LLM generation reinforce these insights: unconstrained models exhibit drift, thematic convergence, and recursive structural decay (Wang, Qianyue et al. 2025; Wang, Wenqing et al. 2025). Even multi-agent (Huot et al. 2024) or hierarchical planning pipelines (Yang et al. 2023) ultimately revert to predictive realization, inheriting hallucination (Banerjee 2024) and limited global control (Peterson 2025). These findings suggest that the absence of structure does not produce creative openness but instead narrows generative possibilities.
Historical narrative systems clarify how constraints can act as generative mechanisms. The Plot Genie formalized the narrative ecology of early twentieth-century American mass-market fiction into an indexed apparatus. As Korda’s Making the List (1999) demonstrates, 1920s–30s bestsellers relied on patterned repertoires—romantic tension, moral conflict, domestic crisis, adventure motifs, and suspense-driven resolution—to stabilize reader expectations and market reliability. The Plot Genie distilled these repertoires into nine indexed categories—settings; male and female protagonists; problems; obstacles to love; complications; predicaments; crises; and climaxes—combining stable narrative roles with designed, rule-governed randomness. Rather than functioning as a static catalog, it offered a proto-computational method balancing stability and variation, aligning with Tenen’s (2024) account of industrial storytelling logics and Hayles’s (1999) pattern/randomness dialectic.
Figure 1: The Craft-in-the-Loop (CITL) generation pipeline.
To ensure that Craft-in-the-Loop (CITL) generates novels at a scale comparable to human-authored fiction, we derived length targets directly from a historical reference corpus. We collected 100 novels from the Publishers Weekly bestseller lists spanning the 1920s and 1930s to establish our reference corpus. Following the acquisition of these texts, we calculated the minimum, maximum, and average token counts to model a normal distribution. To consistently generate full-length novels, we filtered the modeled distribution to exclude lengths below the threshold, which is the average token count of the corpus, and adopted the minimum value from the filtered distribution as our target length. Finally, the derived target is equally divided by the number of chapters to set specific length goals.
Craft-in-the-Loop (CITL) is a multi-agent framework for long-form narrative generation. The Planner agent transforms digitized The Plot Genie indices into a structured chapter plan by augmenting character and situation keywords, deriving stasis and process statements from each dramatic stage, and organizing the resulting actions into hierarchical event trees. These planning artifacts are stored in a shared memory. This memory enables the system to perform selective context retrieval by indexing each artifact to its respective chapter. The Writer agent then retrieves these materials chapter by chapter and expands them into narrative text while maintaining continuity and target length.
The event planning process begins with extracting random narrative keywords (e.g., locale, characters, and dramatic situations) using a computational reproduction of The Plot Genie. However, such keywords are insufficient as seed data for generating a coherent novel. To overcome this limitation, The Planner agent performs a two-step augmentation process. First, the agent expands each character keyword into an augmented profile that includes name, personality, social background, and interpersonal relationship. Second, the agent executes an augmentation to transform the six skeletal dramatic situations — from ‘Problem’ to ‘Climax’ — into a logically connected narrative sequence. Furthermore, to ensure precise structural controllability, the agent executes a two-step refinement process inspired by Chatman’s theory of narrative. Leveraging his insight that stasis statements (static conditions) explicitly ‘project’ process statements (events), the agent first extracts multiple stasis statements from each narrative sequence and then derives corresponding process statements.
Finally, adapting the hierarchical outlining strategy from Detailed Outline Control (Yang et al. 2023), the agent organizes a tree structure. Specifically, process statements serve as the ‘main event’ nodes of an event tree, which the agent expands into distinct ‘sub-event’ nodes to enhance narrative granularity. This procedure is executed systematically across all dramatic situations, resulting in an event tree for each chapter. Upon completion, the agent archives all generated artifacts into the shared memory, indexed by their respective chapter sequence.
For each step in the chapter sequence, the agent retrieves locale and character profiles, target token count, current event tree and chapter sentence from the shared memory. To ensure continuity, the agent also retrieves the full event tree of the previous chapter along with chapter sentences and main events of earlier chapters. This iterative process generates the narrative text for each chapter, sequentially concatenating the outputs to yield the full-length novel.
We adopted GPT-4o for the Planner agent, GPT-5 for the Writer agent. While strict adherence to specific token count constraints remains a challenge across current LLMs, GPT-5 was selected for its superior output capacity, featuring a maximum limit of 128,000 tokens. Regarding hyperparameters, we configured the system to generate five stasis statements per dramatic situation and expand each main event node into three sub-event nodes.
The baselines are defined as follows:
To match the reference corpus, we generated 100 full-length novels (>100,000 tokens) using CITL and Vanilla-GPT. We excluded other existing methods for fair comparison, as they are not optimized for narratives of this volume.
Narrative quality is evaluated across six criteria using a 1-5 point scale: Originality/Creativity, Diversity/Variety, Reader Engagement/Interest, Plot Coherence/Unity, Characters & Setting, and Spatial/Temporal Clarity. For human evaluation, nine expert readers assessed randomly selected 15 sample texts across six criteria using anchored rubrics. Evaluator alignment was maintained through an initial calibration on a shared anchor text followed by post-evaluation reviews, and Label Studio enforced complete ratings and justifications. These samples also served to humanly verify the subsequent computational evaluation, providing an interpretive benchmark against which model-generated scores could be understood.
For computational evaluation, we adopted the G-Eval framework (Liu et al. 2023), which employs LLM-based evaluators to perform step-by-step reasoning. The same six criteria were used, and each prompt specified the criterion definitions, evaluation steps, and scoring rubric to ensure consistent and discriminative scoring across the corpus.
Table 1: Computational evaluation results of CITL compared with Vanilla-GPT and Human-written novels. Boldface indicates the highest score in each category, and underlining indicates the second-highest score.
According to Table 1, CITL significantly outperforms Vanilla-GPT across all metrics. Notably, it surpasses even Human-written novels in 'Originality/Creativity', 'Diversity/Variety', and 'Reader Engagement/Interest'. While Human authors still retain the lead in structural metrics such as 'Plot Coherence/Unity' and 'Spatial/Temporal Clarity', CITL bridges this divide much more effectively than the baseline, proving that a schema-guided approach is essential for the quality required in full-length fiction.
Table 2. Human evaluation results of CITL compared with Vanilla-GPT and Human-written novels. Boldface indicates the highest score in each category, and underlining indicates the second-highest score.
According to Table 2, CITL outperforms Vanilla-GPT across all six criteria in the human evaluation, showing clear improvements in both narrative design and reader-oriented quality. Although Human-written novels still achieve the highest scores overall, CITL comes notably close in key categories such as Reader Engagement/Interest and Characters & Setting. Thus, the human evaluation suggests that CITL does not fully reach the narrative quality of human-written novels, but it meaningfully narrows the gap by improving the structural and descriptive weaknesses most evident in Vanilla-GPT outputs.
The divergence between computational and human evaluation reveals a clear gap between LLM-based metrics and human judgment of narrative quality. While G-Eval suggests that CITL outperforms human-authored novels in Originality/Creativity, Diversity/Variety, and Reader Engagement/Interest, human evaluators consistently rated CITL below human-written novels in the same categories. This discrepancy may reflect the documented self-preference bias of LLM-based evaluators toward LLM-generated text (Liu et al. 2023). Even so, the human evaluation results still demonstrate the efficacy of the CITL framework: the average score gaps between CITL and human-written novels in these criteria remain within 0.3 points, suggesting that CITL substantially bridges the gap between automated generation and human-authored literary quality in key dimensions.
By integrating the indexed structures of Wycliffe A Hill‘s 1931 The Plot Genie with Chatman’s narrative theory within a schema-guided framework, this study serves as an experiment in providing literary theory to make machines write better. The results demonstrate that historically grounded domain knowledge from human experts significantly improves AI-generated fiction: human evaluation confirms consistent gains over the unconstrained baseline, while computational evaluation indicates further strengths without displacing human-written fiction as the upper reference for core structural qualities.