Daejeon, July 27–31
The Yangshi Lei (YSL) architectural archive—recognized in the UNESCO Memory of the World Register—preserves one of the most extensive bodies of premodern Chinese architectural drawings, produced by a hereditary workshop serving the Qing court from the late seventeenth to the early twentieth century (UNESCO World Heritage Centre 2024). Existing scholarship has emphasized that YSL drawings were grounded in inherited Chinese graphic conventions, especially the dipanyang 地盘样, or “ground-plate drawing,” a composite format that assembled column-grid layouts, limited stereoscopic renderings, textual annotations, and landscape-like topographic representations within the same graphic surface (Wu 2004; Zhang 2005; Liu 2006; Wang 2020). These drawings are important not only as records of buildings, but as graphic condensations of Qing official construction techniques, standardized building practices, and workshop-based design knowledge, and cosmological conceptions of space (He 2010; Sun 2022; Wang 2020; Xu 1997).
YSL drawings have more often been interpreted within the continuity of Qing imperial architectural practice than within the industrial and technical contexts of the late-Qing Self-Strengthening Movement (Yangwu yundong 洋务运动). From the 1870s onward, institutions such as the Jiangnan Arsenal translated and circulated technical works that presented mechanical drawing and descriptive geometry as more standardized, coordinate-oriented methods for projecting three-dimensional structures into two-dimensional representations (Zhang 2009; Wang 2016). Several studies suggest that YSL practice was not entirely separate from this technical environment: the workshop has been linked to work associated with the Tianjin Arsenal in 1874 (Li et al. 2023), and Zhang (2005) suggests that the YSL draftsman of the 1875 aerial view of the Hui Tomb may have mastered Western perspective, understood here as central projection. The question is therefore not simply whether YSL drawings were influenced by Western techniques, but how inherited local drawing conventions mediated, translated, and reconfigured the Western projective methods.
To answer this question, projection must be treated not only as a technical method for geometrically representing three-dimensional form, but also as a way of organizing graphic space. Modern visual analysis often assumes either optical coherence, as in perspectival or photographic images, or coordinate coherence, as in standardized engineering drawing. YSL drawings challenge both assumptions. They should not be reduced to optical representation or single-point perspective, since technical and graphic images often organize knowledge through historically specific conventions rather than visual illusion alone (Maynard 2005; Bredekamp / Dünkel / Schneider 2015). Nor should they be measured only against modern projection theory: Wu Cong argues that ancient Chinese architectural drawings transformed three-dimensional objects into two-dimensional equivalents through practical and experiential graphic conventions, rather than through abstract geometric projection alone (Wu 2003). At the same time, architectural representation has long included axonometric, oblique, and other anti-perspectival forms oriented toward constructive clarity rather than optical illusion (Scolari 2012; Lucas 2019). The late-Qing encounter with mechanical drawing and descriptive geometry therefore raises a specific problem: how inherited, composite graphic conventions and Western systematic projective methods coexisted, overlapped, and were reorganized within the same drawing. Identifying these combinations is essential for tracing the graphic transformation of late-Qing architectural practice.
To analyze projection combinations across more than 11,000 YSL drawings, and to extend close reading from selected sheets to archive-scale patterns, automated computer vision techniques become necessary.. Existing AI studies of architectural heritage have adapted established vision architectures—such as YOLO detection, CNN/ResNet/Swin Transformer classification, and attention-based retrieval—to identify building types, façade styles, historical periods, or regional features from remote-sensing images, street views, façade photographs, and image collections (Gao et al. 2023; Gao et al. 2024; Wu et al. 2025). These studies often use medium-sized annotated datasets and benefit from pretrained visual backbones, because their labels are largely grounded in observable building appearance. YSL drawings pose a different problem: architectural objecthood and projection relations are defined by craft knowledge, drawing conventions, and representational logic, rather than directly transferable visual features from optically captured buildings, leading to several challenges in the design of AI technology
The first computational challenge is architectural object recognition in this drawing medium. YSL entities such as platforms, timber structures, topography, and small artifacts may overlap, share linework, or occupy the same visual region while remaining distinct architectural objects. Unlike previous AI studies of architectural heritage, which mainly rely on the optical appearance of buildings for end-to-end recognition, this project addresses a line-based graphic system whose objects are defined by craft knowledge rather than visual continuity alone. It therefore adopts a knowledge-guided detection plus geometric refinement strategy. SAM is useful here because it can generate fine-grained masks from sparse prompts such as boxes and points (Kirillov et al. 2023), making it suitable for refining dense linework and subtle boundaries. However, it cannot by itself determine which knowledge-defined architectural entity should be segmented. Studies in medical imaging and remote sensing show that SAM often requires domain-aware prompting or lightweight adaptation beyond natural images (Huang et al. 2023; Osco et al. 2023), while built-environment applications show that detector-generated box prompts can guide SAM for architectural extraction and floor-plan parsing (Ji et al. 2024; Goonathilake / Thanuja 2024; Akin et al. 2024). Given limited expert annotations, this project therefore adopts a lightweight Detector → SAM strategy: Mask R-CNN (He 2017) is finetuned to learn YSL-specific object categories and convert this domain knowledge into bounding-box prompts, while SAM is kept frozen and used for boundary refinement. Positive and negative point prompts are then designed to further clarify target and competing regions inside ambiguous boxes.
The second computational challenge is projection composition. Previous studies have recognized projection views in standardized engineering drawings, for example by identifying views through geometric relations, recognizing elevations from CAD layers, or reasoning over predefined front, top, and side views for 3D or BIM reconstruction (Lysak 1995; Yin et al. 2020; Hu et al. 2023). However, these methods cannot be directly transferred to YSL drawings, because they assume clear view boundaries, structured CAD information, fixed multi-view layouts, and regular object geometry. YSL drawings are more ambiguous: different projection forms may coexist within one sheet or one architectural group, while object boundaries often overlap with topography, text, and other structures. This project therefore shifts projection analysis from the whole drawing to the segmented object. Each segmented architectural entity becomes a unit for projection regression, symmetry-axis detection, and vanishing-point analysis. The aim is to recover not only what appears in the drawings, but how architectural space is divided, projected, aligned, and made intelligible through drawing.
The source materials are drawn from a digital archive jointly developed by Tianjin University and the National Library of China. From this archive, 60 representative drawings out of 11,117 were manually annotated with object categories and projection information. Object categories include topography, brick and stone structures, timber buildings, small architectural artifacts, and sky. Projection types are defined with reference to ISO 128 and ISO 5456, including plan, elevation, axonometric, oblique, auxiliary, and locally symmetric projection forms (International Organization for Standardization 1996–2011a; International Organization for Standardization 1996–2011b). These categories do not assume that YSL drawings followed modern technical standards; rather, they provide an operational vocabulary for comparing inherited YSL graphic conventions with Western-derived projective methods.
Figure 1. Annotation of representative YSL drawings, colored by instance (left: NLC349-1260, right: NLC161-006).
First, we finetune a pretrained Mask R-CNN network on the annotated drawings to predict object categories, bounding boxes, and coarse masks (He et al. 2017). During inference, each predicted bounding box is used as a primary prompt for SAM: the box tells SAM the approximate region in which an architectural object is located. SAM is used off-the-shelf as a boundary-refinement model and is not finetuned. Preliminary results show stable high-confidence predictions on annotated training examples (Figure 2). On held-out drawings, retaining some lower-confidence Mask R-CNN predictions may help preserve plausible candidate boxes for SAM refinement. As the training set expands through data augmentation (generating variations of existing drawings) and model-assisted annotation (where the AI suggests labels that are then corrected by humans), we will improve the model model's accuracy and confidence on held-out drawings.
The next planned refinement step will add positive and negative point prompts inside each box. Positive points indicate reliable interior regions of the target object, whereas negative points mark nearby structures that do not belong to the target object. The point prompts reduce ambiguity when multiple architectural elements appear in the same box.
Figure 2. Preliminary results of the detector-guided SAM refinement step on annotated training examples. Red boxes show Mask R-CNN proposals, and purple regions show SAM-generated masks. The examples illustrate how the pipeline extracts candidate masks for timber buildings, brick and stone structures, and topography for subsequent projection analysis.
Second, we estimate the projection techniques used in the drawing of architectural structures by performing projection regression on the segmented objects. While projection categories provide the baseline annotation framework, projection itself is treated as a continuous variable, allowing the model to estimate gradual differences in viewing direction, viewing height, and obliqueness rather than enforcing rigid labels. An exploratory test using SAM-generated masks and mask-level image embeddings showed promising results for continuous projection regression with lightweight MLP heads. The planed pipeline extends this result by using Mask R-CNN-guided SAM refinement to obtain more stable architectural object units before projection analysis.
Finally, object-level segmentation and projection results will be integrated at the full-image level. Boundary structures and projection directions will be used to infer symmetry axes and vanishing-point structures through geometric methods such as PCA and RANSAC (Fischler / Bolles 1981; Jolliffe 2011). In some late-Qing construction drawings, symmetric oblique projections generate local vanishing points; when multiple projected units align along a central axis, these local vanishing points form vanishing-point chains that visually guide attention and organize construction sequences (Figure 3). These object- and sheet-level measurements can then be aggregated by time period and architectural categories, allowing the project to trace changes in projection forms, geometric symmetry, and construction-oriented visual organization across the archive.
Figure 3. Vanishing-point chains in the partial drawing of NLC 223-007.
With this paper, we propose a detector-guided segmentation and projection-analysis framework for studying the Yangshi Lei archive as a system of visual-spatial reasoning. Our first contribution is methodological: it adapts pretrained vision models to historical architectural drawings whose objects and projection relations are defined by craft knowledge rather than visual continuity alone, and are therefore not directly available in general visual pretraining. By using Mask R-CNN for architectural localization and SAM for boundary refinement, the pipeline turns complex drawing regions into analyzable object units.
Our second contribution is historical and representational: it treats projection not as a sheet-level label, but as an object-level feature through which multiple drawing conventions can coexist within one image. This makes it possible to study how inherited YSL graphic practices and Western geometrized drawing techniques may have interacted in late-Qing architectural representation.
At the current stage, the project has produced annotated training data, preliminary Mask R-CNN→SAM segmentation results, and promising exploratory results for continuous projection regression. The next stages will expand annotation through augmentation and expert correction of model-generated labels, evaluate the pipeline on held-out drawings, incorporate positive/negative point prompts, and aggregate projection, symmetry, and vanishing-point features across time periods and architectural categories. In doing so, the project aims to make non-optical and multi-projection drawing systems computationally traceable without reducing them to modern photographic or perspectival models.