Daejeon, July 27–31
Early toponym recognition in ancient Chinese literature relied mainly on manual rules and lexicon-based methods[1]. However, these methods suffered from narrow coverage, rigid patterns, and costly rule design, lacking robustness, portability, and comprehensiveness[2]. Subsequently, BiLSTM and CRF later improved performance and generalization, with BiLSTM-CRF excelling at entity recognition in classical medical texts[3] and botanical entity recognition[4]. However, these methods still have limitations in fully representing the contextual semantics of historical texts.
The breakthrough came with BERT. BERT’s bidirectional attention, combined with BiLSTM and CRF, forms a robust, cross-domain adaptable architecture[5][6]. In ancient Chinese texts, BERT-BiLSTM-CRF proved highly effective for NER tasks such as in the Twenty-Four Histories[7] and processing historical documents[8]. However, Ancient Chinese text deep learning is challenged by scarce data, unique language, and complex toponyms. Local feature fusion offers a partial solution[9], position embedding[10], multi-stage fine-tuning[11] and regularity-guided boundary detection[12] to address these issues and bridge the domain gap.
We compared CNN, LSTM, and BERT and their combinations for Chinese toponym recognition. The main contributions are: (1) developing a BERT-BiLSTM-CRF model for location name identification; (2) comparing its performance with other models; and (3) identifying optimal parameters via systematic tuning to guide similar toponym recognition tasks.
As shown in Figure 1, the research process includes: collecting ancient texts, preprocessing (tokenization, segmentation, label alignment), training with BERT-BiLSTM-CRF, evaluating with Precision/Recall/F1, and visualizing results.
Figure 1 Flowchart for Ancient Chinese Toponym Recognition in Ancient Texts.
The main data source is the digitized Comprehensive Chronological Biography of Wang Yangming (OCR to TXT). Using the BIO system, place names were annotated character-by-character (B-LOC: start, I-LOC: middle/end, O: non-place-name), including punctuation. A total of 462 TXT files were annotated and randomly split into training (70%), validation (15%), and test (15%) sets.
This study uses Google’s Chinese-BERT-wwm to capture semantic and lexical features in ancient Chinese texts via bidirectional context and whole word masking. BiLSTM uses two opposite-direction LSTM networks to handle word sequences, meaning shifts, and long-distance dependencies, improving toponym recognition in complex texts. CRF optimizes sequence labeling by modeling label transition probabilities. The proposed BERT-BiLSTM-CRF model (Figure 2) integrates BERT for semantic extraction, BiLSTM for dynamic context, and CRF for label optimization to output toponym recognition results.
Figure 2 The Model Structure of BERT-BiLSTM-CRF.
To verify the effectiveness of the model, this study conducted a series of experiments. The specific environmental configurations and training parameter settings are presented in Table 1 and Table 2, respectively.
Table 1 Experimental Environment Configuration
| Environment Name | Configuration |
| Operating System | Windows10 |
| CPU | Intel(R) Core(TM) i7-10875H |
| GPU | NVDIA Geforce RTX2060 Laptop |
| RAM | 16G |
| Pytorch | 2.6.0 |
| CUDA | 11.8 |
Table 2 Experimental Training Parameters
| Training Parameter Name | Parameter Value | Explanation of Parameter Name |
| batch size | 16 | The number of training samples per batch |
| epochs | 30 | The number of training rounds |
| learning rate | 3e-5 | Learning rate |
| max length | 128 | Maximum sequence length |
| optimizer | Adam | Optimizer |
To validate the proposed model, multiple models were compared in NER experiments using F1-score, precision, and recall. As shown in Table 3, the BERT-BiLSTM-CRF model achieves the highest precision, recall, and F1-score among all compared models. Table 4 further illustrates that BERT-BiLSTM-CRF accurately identifies toponyms while other models produce false positives (e.g., identifying "室" or "久" as toponyms) or false negatives.
Table 3 Comparison of different models (%)
| Model | F1-score | Precision | Recall |
| BERT-BiLSTM-CRF | 0.897 | 0.875 | 0.920 |
| BERT | 0.861 | 0.821 | 0.905 |
| BERT-BiLSTM | 0.854 | 0.815 | 0.815 |
| BERT-CRF | 0.874 | 0.864 | 0.883 |
| BiLSTM-CRF | 0.753 | 0.814 | 0.700 |
Table 4 Examples of model performance in recognition tasks
| Original Text | BERT-BiLSTM-CRF | BERT | BERT-BiLSTM | BERT-CRF | BiLSTM-CRF |
| 归至【绍兴】。筑室【阳明洞】中，行导引术，静坐习定，究极道经秘旨。 | 绍兴，阳明洞 | 绍兴，阳明洞 | 绍兴，室，阳明洞 | 绍兴，阳明洞 | 绍兴，阳明洞，秘旨 |
| 上处置平复地方以图久安疏，奏请立土官知州，立岑邦相于【田州】，岑邦佐于【武靖】 | 田州、武靖 | 久安、田州、武靖 | 田州、武靖 | 久，田州，武靖 | 田州 |
To systematically examine the effects of training epochs and dropout rate on model performance, a 5×5 grid search was performed using five dropout rates (0.1-0.5) and five epoch values (10-50), with all other hyperparameters fixed. Early stopping (patience=5) was applied based on validation F1 score. the results are shown in Figure 3.
Figure 3 Impact of Epoch and Dropout Rate on Model Performance.
The figure shows that all curves converge around 20 epochs. Moderate dropout (0.3–0.4) improves generalization; low dropout (0.1) gives fast early gains but later stagnates; high dropout (0.5) lowers F1 scores.
Overall, The best performance (F1 = 0.906) occurs at Epoch=20, Dropout=0.1, a 0.9% gain over the baseline (0.897). This optimal combination balances training duration and regularization, serving as a reference for fine-tuning similar models in ancient text digitization and historical GIS.
BERT-BiLSTM-CRF outperformed BERT, BERT-BiLSTM, and BERT-CRF in ancient text NER by combining context-aware representations, long-range dependency handling, and label optimization. The poor performance of BiLSTM-CRF highlights the importance of pre-trained models for specialized domains.
Parameter tuning for BERT-BiLSTM-CRF showed that the optimal combination (Epoch=20, Dropout=0.1) improved the F1 score, highlighting the importance of balancing training duration and regularization. Performance gains may be limited by the small training dataset. Future work should use larger, more diverse datasets to further optimize fine-tuning.
This study proposes BERT-BiLSTM-CRF for ancient text NER. Empirical results confirm its effectiveness over baseline models, and parameter tuning provides insights into model optimization. However, several limitations should be acknowledged: the small dataset limits learning of complex patterns; evaluation focuses exclusively on F1, precision, and recall, overlooking interpretability and efficiency; and the method is tested only on ancient text NER without extension to broader historical or cultural applications. Future work should therefore prioritize data expansion, metric diversification, and application generalization. Overall, this work contributes a robust baseline for ancient text information extraction.
Acknowledgements
This work is supported by the separate project on Chinese studies in the Philosophy and Social Sciences Planning of Guizhou Province (22GZGX03).