Please wait a minute...
Data Analysis and Knowledge Discovery  2021, Vol. 5 Issue (9): 63-74    DOI: 10.11925/infotech.2096-3467.2021.0460
Current Issue | Archive | Adv Search |
Annotation Method for Extracting Entity Relationship from Ancient Chinese Works
Wang Yifan1,Li Bo2,Shi Hua3,Miao Wei1(),Jiang Bin2
1School of Northeast Asia Studies, Shandong University, Weihai 264209, China
2School of Mechanical, Electrical & Information Engineering, Shandong University, Weihai 264209, China
3School of Transborder Studies, Arizona State University, Tucson 85257, USA
Download: PDF (916 KB)   HTML ( 4
Export: BibTeX | EndNote (RIS)      

[Objective] This paper proposes an annotation method for ancient Chinese datasets, aiming to standardize the annotation procedures. [Objective] We proposed a new method integrating logical semantics, deep learning and history knowledge. This model, which is suitable for few-shot learning, includes three principles of “annotation of relationship valence”, “annotation of propositional logic”, “existence of a single relationship”. [Results] We examined the proposed annotation model with the text dataset of Shiji (Historical Records in Chinese), and found its F1 values for the tasks of relationship extraction and the propositional logic extraction reached 42.02% and 34.07% respectively. [Limitations] The proposed method, which did not include the pre-trained models like BERT or ALBERT, only used the classic Word2Vec model for word embedding. The model's performance could be further improved. [Conclusions] Our new annotation method could effectively extract entity relationship from Ancient Chinese works.

Key wordsNatural Language Processing      Relation Extraction      Sequence Tagging      Shiji     
Received: 10 May 2021      Published: 15 October 2021
ZTFLH:  分类号: TP391  
Fund:*National Social Science Fund of China(17VGB005)
Corresponding Authors: Miao Wei     E-mail:

Cite this article:

Wang Yifan,Li Bo,Shi Hua,Miao Wei,Jiang Bin. Annotation Method for Extracting Entity Relationship from Ancient Chinese Works. Data Analysis and Knowledge Discovery, 2021, 5(9): 63-74.

URL:     OR

名称 标记 意义
生命/Life LFE 任何与创造、伤害甚至毁灭生命有关的语句核心
社交/Social Contact SCT 任何社交类语句核心
空间/Location LOC 任何有关空间位置的语句核心
政治/Politics POL 任何政治性的语句核心
动作/Action Towards Object ATO 任何与创造、伤害甚至毁灭非生命有关的语句核心
战争/War WAR 任何与战争有关的语句核心
Annotation Method for Parent Class
名称 标记 对照设置(子类别数量/总数)
生命/Life LFE 36/209
社交/Social Contact SCT 49/85
空间/Location LOC 88/171
政治/Politics POL 437/789
动作/Action Towards Object ATO 74/126
战争/War WAR 23/36
Control Group for Parent Class
关系类型 主体 受体 标注位置 原始数据
空间/LOC 黄帝 轩辕之丘 黄帝居轩辕之丘
并列/SJ-BL ,而 黄帝居轩辕之丘,而娶于西陵之女
并列/SJ-BL ,而 黄帝居轩辕之丘,而娶于西陵之女
社交/SCT 黄帝 西陵之女 娶于 黄帝居轩辕之丘,而娶于西陵之女
属性/SX 西陵之女 嫘祖 是为 而娶于西陵之女,是为嫘祖
Example of the Annotation Method for Joint Extraction of Entity and Relationship in Ancient Chinese Field
The Annotation Method for Joint Extraction of Entity and Relationship in Ancient Chinese Field
The Word Embedding-BiGRU-CRF Model
类型 标注数量 类型 标注数量
主体/SBJ 3 283 受体/OBJ 2 328
生命/LFE 209 属性/SX 157
社交/SCT 85 事件属性/SJSX 440
空间/LOC 171 递进/SJ-DJ 810
政治/POL 789 并列/SJ-BL 140
动作/ATO 126 因果/SJ-YG 57
战争/WAR 36 转折/SJ-ZZ 20
Type and Quantity of the Relationship Data
关系类型 准确率 召回率 F1值
SBJ 65.68% 65.88% 65.78%
OBJ 53.29% 43.55% 47.93%
LFE 90.00% 75.00% 81.82%
SCT 16.67% 7.69% 10.53%
LOC 29.63% 20.00% 23.88%
POL 33.57% 36.36% 34.91%
ATO 0.00% 0.00% 0.00%
WAR 71.43% 50.00% 58.82%
SX 56.41% 53.66% 55.00%
SJSX 18.92% 9.21% 12.39%
整体结果 48.10% 38.38% 42.02%
Training Results of Relationship Extraction Model
关系类型 标注条数 子类别数量
SCT 85 52
ATO 126 87
LOC 171 96
POL 789 485
SJSX 440 396
DJ 810 52
BL 140 10
YG 57 16
ZZ 20 6
Number of Propositional Logic Annotations and Subclass
命题逻辑 准确率 召回率 F1
DJ 60.38% 57.55% 58.93%
BL 67.35% 27.73% 39.29%
YG 28.57% 12.50% 17.39%
ZZ 0.00% 0.00% 0.00%
整体结果 42.79% 30.35% 34.07%
Training Results of Propositional Logic Extraction Model
[1] 石民, 李斌, 陈小荷. 基于CRF的先秦汉语分词标注一体化研究[J]. 中文信息学报, 2010, 24(2):39-45.
[1] ( Shi Min, Li Bin, Chen Xiaohe. CRF Based Research on a Unified Approach to Word Segmentation and POS Tagging for Pre-Qin Chinese[J]. Journal of Chinese Information Processing, 2010, 24(2):39-45.)
[2] 陈小荷. 先秦文献信息处理[M]. 北京: 世界图书出版公司, 2013.
[2] ( Chen Xiaohe. Information Processing of Documents of Pre-Qin Period[M]. Beijing: Word Publishing Corporation, 2013.)
[3] 严顺. 基于CRF的古汉语分词标注模型研究[J]. 江苏科技信息, 2016(8):10-12.
[3] ( Yan Shun. Research on Word Segmentation and Tagging for Ancient Chinese Based on CRF[J]. Jiangsu Science & Technology Information, 2016(8):10-12.)
[4] 王晓玉, 李斌. 基于CRFs和词典信息的中古汉语自动分词[J]. 数据分析与知识发现, 2017, 1(5):62-70.
[4] ( Wang Xiaoyu, Li Bin. Automatically Segmenting Middle Ancient Chinese Words with CRFs[J]. Data Analysis and Knowledge Discovery, 2017, 1(5):62-70.)
[5] 杨世超, 纪月, 赵立鹏. 基于条件随机场的古汉语分词研究[J]. 电脑知识与技术, 2017, 13(22):183-184.
[5] ( Yang Shichao, Ji Yue, Zhao Lipeng. Study of Ancient Chinese Word Segmentation Based on Conditional Random Field[J]. Computer Knowledge and Technology, 2017, 13(22):183-184.)
[6] 俞敬松, 魏一, 张永伟, 等. 基于非参数贝叶斯模型和深度学习的古文分词研究[J]. 中文信息学报, 2020, 34(6):1-8.
[6] ( Yu Jingsong, Wei Yi, Zhang Yongwei, et al. Word Segmentation for Ancient Chinese Texts Based on Nonparametric Bayesian Models and Deep Learning[J]. Journal of Chinese Information Processing, 2020, 34(6):1-8.)
[7] 黄水清, 王东波, 何琳. 基于先秦语料库的古汉语地名自动识别模型构建研究[J]. 图书情报工作, 2015, 59(12):135-140.
[7] ( Huang Shuiqing, Wang Dongbo, He Lin. Research on Constructing Automatic Recognition Model for Ancient Chinese Place Names Based on Pre-Qin Corpus[J]. Library and Information Service, 2015, 59(12):135-140.)
[8] 崔丹丹, 刘秀磊, 陈若愚, 等. 基于Lattice LSTM的古汉语命名实体识别[J]. 计算机科学, 2020, 47(S2):18-22.
[8] ( Cui Dandan, Liu Xiulei, Chen Ruoyu, et al. Named Entity Recognition in Field of Ancient Chinese Based on Lattice LSTM[J]. Computer Science, 2020, 47(S2):18-22.)
[9] Marrero M, Urbano J, Sánchez-Cuadrado S, et al. Named Entity Recognition: Fallacies, Challenges and Opportunities[J]. Computer Standards & Interfaces, 2013, 35(5):482-489.
doi: 10.1016/j.csi.2012.09.004
[10] Kumar S. A Survey of Deep Learning Methods for Relation Extraction[OL]. arXiv Preprint, arXiv: 1705.03645.
[11] Zheng S, Wang F, Bao H, et al. Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme [C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017.
[12] 曹明宇, 杨志豪, 罗凌, 等. 基于神经网络的药物实体与关系联合抽取[J]. 计算机研究与发展, 2019, 56(7):1432-1440.
[12] ( Cao Mingyu, Yang Zhihao, Luo Ling, et al. Joint Drug Entities and Relations Extraction Based on Neural Networks[J]. Journal of Computer Research and Development, 2019, 56(7):1432-1440.)
[13] Zeng X R, Zeng D J, He S Z, et al. Extracting Relational Facts by an End-to-End Neural Model with Copy Mechanism [C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2018: 506-514.
[14] Devlin J, Chang M, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding [C]//Proceedings of 2019 Conference of the North American Chapter of the Association for Computational Linguistics. 2019: 4171-4186.
[15] Lan Z Z, Chen M D, Goodman S, et al. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations [C]//Proceedings of the International Conference on Learning Representations. 2020.
[16] 唐晓波, 刘志源. 金融领域文本序列标注与实体关系联合抽取研究[J]. 情报科学, 2021, 39(5):3-11.
[16] ( Tang Xiaobo, Liu Zhiyuan. Research on Text Sequence Tagging and Joint Extraction of Entity and Relation in Financial Field[J]. Information Science, 2021, 39(5):3-11.)
[17] 吴赛赛, 梁晓贺, 谢能付, 等 面向领域实体关系联合抽取的标注方法[J]. 计算机应用, .
[17] ( Wu Saisai, Liang Xiaohe, Xie Nengfu, et al. Annotation Method for Joint Extraction of Domain-Oriented Entity and Relation. Journal of Computer Applications )
[18] 张坤丽, 赵旭, 关同峰, 等. 面向医疗文本的实体及关系标注平台的构建及应用[J]. 中文信息学报, 2020, 34(6):36-44.
[18] ( Zhang Kunli, Zhao Xu, Guan Tongfeng, et al. A Platform for Entity and Entity Relationship Labeling in Medical Texts[J]. Journal of Chinese Information Processing, 2020, 34(6):36-44.)
[19] Montague R. Universal Grammar[J]. Theoria, 1970, 36(3):373-398.
doi: 10.1111/(ISSN)1755-2567
[20] 朱水林. 逻辑语义学研究[M]. 上海: 上海教育出版社, 1992.
[20] ( Zhu Shuilin. The Study of Logical Semantics[M]. Shanghai: Shanghai Educational Publishing House, 1992.)
[21] 周国光, 张国宪. 汉语的配价语法理论研究[J]. 语文建设, 1994(9):33-36.
[21] ( Zhou Guoguang, Zhang Guoxian. Study on the Theory of Valence Grammar in Chinese[J]. Language Planning, 1994(9):33-36.)
[22] 周国光. 汉语配价语法论略[J]. 南京师大学报(社会科学版), 1994(4):103-106, 121.
[22] ( Zhou Guoguang. A Brief Study on Chinese Valence Grammar[J]. Journal of Nanjing Normal University(Social Science Edition), 1994(4):103-106, 121.)
[23] 靳光瑾, 陆汝占. 从汉语句子中提取逻辑函子的一种方法[J]. 软件学报, 1998, 9(6):444-447.
[23] ( Jin Guangjin, Lu Ruzhan. A Method for Extracting Logical Functors from Chinese Sentences[J]. Journal of Software, 1998, 9(6):444-447.)
[24] 司马迁. 史记[M]. 北京: 中华书局, 1959.
[24] ( Sima Qian. Shiji[M]. Beijing: Zhonghua Book Company, 1959.)
[25] 李冬梅, 张扬, 李东远, 等. 实体关系抽取方法研究综述[J]. 计算机研究与发展, 2020, 57(7):1424-1448.
[25] ( Li Dongmei, Zhang Yang, Li Dongyuan, et al. Review of Entity Relation Extraction Methods[J]. Journal of Computer Research and Development, 2020, 57(7):1424-1448.)
[26] Mikolov T, Chen K, Corrado G S, et al. Efficient Estimation of Word Representations in Vector Space [C]//Proceedings of the 1st International Conference on Learning Representations. 2013.
[27] Miwa M, Bansal M. End-to-End Relation Extraction Using LSTMs on Sequences and Tree Structures [C] //Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016: 1105-1116.
[28] Pennington J, Socher R, Manning C. GloVe: Global Vectors for Word Representation [C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014: 1532-1543.
[29] Levy O, Goldberg Y, Dagan I. Improving Distributional Similarity with Lessons Learned from Word Embeddings[J]. Transactions of the Association for Computational Linguistics, 2015, 3:211-225.
doi: 10.1162/tacl_a_00134
[30] Cho K, van Merrienboer B, Gulcehre C, et al. Learning Phrase Representations Using RNN Encoder-decoder for Statistical Machine Translation[OL]. arXiv Preprint, arXiv: 1406.1078.
[31] Lafferty J D, McCallum A, Pereira F C N. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data [C]//Proceedings of the 18th International Conference on Machine Learning. 2001: 282-289.
[32] Nakagawa T, Inui K, Kurohashi S, et al. Dependency Tree-based Sentiment Classification Using CRFs with Hidden Variables [C]//Proceedings of Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. 2010: 786-794.
[33] Bahdanau D, Cho K, Bengio Y, et al. Neural Machine Translation by Jointly Learning to Align and Translate [C]//Proceedings of the 3rd International Conference on Learning Representations. 2015.
[1] Ma Jiangwei, Lv Xueqiang, You Xindong, Xiao Gang, Han Junmei. Extracting Relationship Among Military Domains with BERT and Relation Position Features[J]. 数据分析与知识发现, 2021, 5(8): 1-12.
[2] Mingxuan Huang,Shoudong Lu,Hui Xu. Cross-Language Information Retrieval Based on Weighted Association Patterns and Rule Consequent Expansion[J]. 数据分析与知识发现, 2019, 3(9): 77-87.
[3] Jiahui Hu,An Fang,Wanqing Zhao,Chenliu Yang,Huiling Ren. Annotating Chinese E-Medical Record for Knowledge Discovery[J]. 数据分析与知识发现, 2019, 3(7): 123-132.
[4] Zhang Qin,Guo Hongmei,Zhang Zhixiong. Extracting Entity Relationship with Word Embedding Representation Features[J]. 数据分析与知识发现, 2017, 1(9): 8-15.
[5] Yang Chunlei. Quantification Constraint System for Pragmatic Disambiguation: From Linguistic Design to Computational Implementation[J]. 数据分析与知识发现, 2017, 1(11): 1-11.
[6] Liu Jianhua,Wang Ying,Zhang Zhixiong,Li Chuanxi. Extracting Semantic Knowledge from Plant Species Diversity Collections[J]. 数据分析与知识发现, 2017, 1(1): 37-46.
[7] Yang Chunlei. Building Online System for Chinese Lexicon and Grammar[J]. 现代图书情报技术, 2016, 32(7-8): 129-136.
[8] Liu Tianyi,Bu Yi Zhao Danqun Huang Wenbin,Zhao Danqun,Huang Wenbin. Review of Citation-based Automatic Summarization Studies[J]. 现代图书情报技术, 2016, 32(5): 1-8.
[9] Peng Hao, Xu Jian, Xiao Zhuo. Sentiment Analysis of Web Reviews Based on Comparative Sentence Extraction[J]. 现代图书情报技术, 2015, 31(12): 48-56.
[10] Yang Chunlei, Dan Flickinger. ManGO:Grammar Engineering for Deep Linguistic Processing[J]. 现代图书情报技术, 2014, 30(3): 57-64.
[11] Qiu Junping, Fang Guoping. The Comparative Analysis of Natural Language Processing Research at Home and Abroad Based on Knowledge Mapping[J]. 现代图书情报技术, 2014, 30(12): 51-61.
[12] She Guiqing, Zhang Yongan. Study on the Model of Automatic Extraction and Annotation of Trail Cases[J]. 现代图书情报技术, 2013, (6): 23-29.
[13] Wang Xiuyan, Cui Lei. Extract Semantic Relations Between Biomedical Entities Applied Hybrid Method[J]. 现代图书情报技术, 2013, 29(3): 77-82.
[14] Huang Xun, You Hongliang, Yu Yang. A Review of Relation Extraction[J]. 现代图书情报技术, 2013, 29(11): 30-39.
[15] Wang Xiuyan, Cui Lei. Overview of Semantic Relations Extraction Between Biomedical Entities by Key Verbs[J]. 现代图书情报技术, 2011, 27(9): 21-27.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938