|
|
|
| Extracting Relationship Among Military Domains with BERT and Relation Position Features |
Ma Jiangwei1,Lv Xueqiang1,You Xindong1( ),Xiao Gang2,Han Junmei2 |
1Beijing Key Laboratory of Internet Culture and Digital Dissemination Research, Beijing Information Science and Technology University, Beijing 100101, China
2National Key Laboratory of Complex System Simulation, Beijing 100101, China |
|
|
|
|
Abstract [Objective] This article addresses the difficulties of relationship extraction due to overlapping entity relationship in military texts. [Methods] We used the BERT model as the encoder for the input texts, and used the hierarchical reinforcement learning approach to decode relationship and their corresponding entities. Then, we merged the relational position features in the entity decoding process to construct a relationship extraction model for military domains. [Results] The F1 value reached 82.2% on the military weapon and equipment dataset, which was about 8% higher than other methods. Using the publicly available NYT10 and NYT10-sub datasets, the F1 values reached 71.8% and 69.0%, which was about 7% and 9% higher than other methods. [Limitations] The new method’s extraction performance is better on manually annotated datasets. More research is needed to improve it performance on remotely supervised datasets with much noise. [Conclusions] The HBP method could effectively extract relationship among the military domains, and has some generalization potentiality.
|
|
Received: 24 February 2021
Published: 15 September 2021
|
|
|
| Fund:Natural Science Foundation of Beijing(4212020);Defense-related Science and Technology Key Lab Fund Project(6412006200404);Qin Xin Talents Cultivation Program, Beijing Information Science & Technology University(QXTCP B201908) |
|
Corresponding Authors:
You Xindong ORCID：0000-0002-3351-4599
E-mail: youxindong@bistu.edu.cn
|
| [1] |
林旺群, 汪淼, 王伟, 等. 知识图谱研究现状及军事应用[J]. 中文信息学报, 2020, 34(12):9-16.
|
| [1] |
( Lin Wangqun, Wang Miao, Wang Wei, et al. A Survey to Knowledge Graph and Its Military Application[J]. Journal of Chinese Information Processing, 2020, 34(12):9-16.)
|
| [2] |
鄂海红, 张文静, 肖思琪, 等. 深度学习实体关系抽取研究综述[J]. 软件学报, 2019, 30(6):1793-1818.
|
| [2] |
( E Haihong, Zhang Wenjing, Xiao Siqi, et al. Survey of Entity Relationship Extraction Based on Deep Learning[J]. Journal of Software, 2019, 30(6):1793-1818.)
|
| [3] |
Mintz M, Bills S, Snow R, et al. Distant Supervision for Relation Extraction without Labeled Data [C]// Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. 2009: 1003-1011.
|
| [4] |
Gormley M R, Yu M, Dredze M. Improved Relation Extraction with Feature-Rich Compositional Embedding Models [C]// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015: 1774-1784.
|
| [5] |
Zheng S C, Wang F, Bao H Y, et al. Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme [C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017: 1227-1236.
|
| [6] |
Takanobu R, Zhang T Y, Liu J X, et al. A Hierarchical Framework for Relation Extraction with Reinforcement Learning [C]// Proceedings of the AAAI Conference on Artificial Intelligence. 2019: 7072-7079.
|
| [7] |
Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding [C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019: 4171-4186.
|
| [8] |
Lample G, Ballesteros M, Subramanian S, et al. Neural Architectures for Named Entity Recognition [C]// Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2016: 260-270.
|
| [9] |
Santos C N D, Xiang B, Zhou B W. Classifying Relations by Ranking with Convolutional Neural Networks [C]// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 2015: 626-634.
|
| [10] |
张琴, 郭红梅, 张智雄. 融合词嵌入表示特征的实体关系抽取方法研究[J]. 数据分析与知识发现, 2017, 1(9):8-15.
|
| [10] |
( Zhang Qin, Guo Hongmei, Zhang Zhixiong. Extracting Entity Relationship with Word Embedding Representation Features[J]. Data Analysis and Knowledge Discovery, 2017, 1(9):8-15.)
|
| [11] |
张东东, 彭敦陆. ENT-BERT:结合BERT和实体信息的实体关系分类模型[J]. 小型微型计算机系统, 2020, 41(12):2557-2562.
|
| [11] |
( Zhang Dongdong, Peng Dunlu. ENT-BERT: Entity Relation Classification Model Combining BERT and Entity Information[J]. Journal of Chinese Computer Systems, 2020, 41(12):2557-2562.)
|
| [12] |
李卫疆, 李涛, 漆芳. 基于多特征自注意力BLSTM的中文实体关系抽取[J]. 中文信息学报, 2019, 33(10):47-56.
|
| [12] |
( Li Weijiang, Li Tao, Qi Fang. Chinese Entity Relation Extraction Based on Multi-Features Self-Attention Bi-LSTM[J]. Journal of Chinese Information Processing, 2019, 33(10):47-56.)
|
| [13] |
吴粤敏, 丁港归, 胡滨. 基于注意力机制的农业金融文本关系抽取研究[J]. 数据分析与知识发现, 2019, 3(5):86-92.
|
| [13] |
( Wu Yuemin, Ding Ganggui, Hu Bin. Extracting Relationship of Agricultural Financial Texts with Attention Mechanism[J]. Data Analysis and Knowledge Discovery, 2019, 3(5):86-92.)
|
| [14] |
Li Q, Ji H. Incremental Joint Extraction of Entity Mentions and Relations [C]// Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. 2014: 402-412.
|
| [15] |
Ren X, Wu Z Q, He W Q, et al. CoType: Joint Extraction of Typed Entities and Relations with Knowledge Bases [C]// Proceedings of the 26th International Conference on World Wide Web. 2017: 1015-1024.
|
| [16] |
Miwa M, Bansal M. End-to-End Relation Extraction Using LSTMs on Sequences and Tree Structures [C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016: 1105-1116.
|
| [17] |
Bekoulis G, Deleu J, Demeester T, et al. Joint Entity Recognition and Relation Extraction as a Multi-head Selection Problem[J]. Expert Systems with Applications, 2018, 114:34-45.
doi: 10.1016/j.eswa.2018.07.032
|
| [18] |
Li X Y, Yin F, Sun Z J, et al. Entity-Relation Extraction as Multi-Turn Question Answering [C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 1340-1350.
|
| [19] |
Wei Z P, Su J L, Wang Y, et al. A Novel Cascade Binary Tagging Framework for Relational Triple Extraction [C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 1476-1488.
|
| [20] |
Zeng D J, Liu K, Lai S, et al. Relation Classification via Convolutional Deep Neural Network [C]// Proceedings of the 25th International Conference on Computational Linguistics. 2014: 2335-2344.
|
| [21] |
Ji G, Liu K, He S W, et al. Distant Supervision for Relation Extraction with Sentence-level Attention and Entity Descriptions [C]// Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2017: 3060-3066.
|
| [22] |
李芊芊, 张克亮. 基于依存分析的军事领域英文实体关系抽取研究[J]. 情报工程, 2019, 5(1):98-112.
|
| [22] |
( Li Qianqian, Zhang Keliang. Entity Relation Extraction Based on Dependency Parsing in Military Field[J]. Technology Intelligence Engineering, 2019, 5(1):98-112.)
|
| [23] |
田佳来, 吕学强, 游新冬, 等. 基于分层序列标注的实体关系联合抽取方法[J]. 北京大学学报(自然科学版), 2021, 57(1):53-60.
|
| [23] |
( Tian Jialai, Lü Xueqiang, You Xindong, et al. Joint Extraction of Entities and Relations Based on Hierarchical Sequence Labeling[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(1):53-60.)
|
| [24] |
Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need[J]. Advances in Neural Information Processing Systems, 2017, 30:5998-6008.
|
| [25] |
李培芸, 李茂西, 裘白莲, 等. 融合BERT语境词向量的译文质量估计方法研究[J]. 中文信息学报, 2020, 34(3):56-63.
|
| [25] |
( Li Peiyun, Li Maoxi, Qiu Bailian, et al. Integrating BERT Word Embedding into Quality Estimation of Machine Translation[J]. Journal of Chinese Information Processing, 2020, 34(3):56-63.)
|
| [26] |
Sutton R S, Barto A G. Reinforcement Learning: An Introduction[M]. MIT Press, 1998.
|
| [27] |
Bishop C M. Pattern Recognition and Machine Learning[M]. Springer, 2006.
|
| [28] |
van Rijsbergen C J. Information Retrieval[M]. Butterworths, 1975.
|
| [29] |
Sutton R S, McAllester D A, Singh S P, et al. Policy Gradient Methods for Reinforcement Learning with Function Approximation[A]// Advances in Neural Information Processing Systems[M]. MIT Press, 2000: 1057-1063.
|
| [30] |
Riedel S, Yao L M, McCallum A. Modeling Relations and Their Mentions without Labeled Text [C]// Proceedings of Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, Berlin, Heidelberg, 2010: 148-163.
|
| [31] |
Zeng X R, Zeng D J, He S Z, et al. Extracting Relational Facts by an End-to-end Neural Model with Copy Mechanism [C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018: 506-514.
|
| [32] |
Fu T J, Li P H, Ma W Y. GraphRel: Modeling Text as Relational Graphs for Joint Entity and Relation Extraction [C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 1409-1418.
|
| [33] |
Su J L. Hybrid Structure of Pointer and Tagging for Relation Extraction: A Baseline[EB/OL].(2019-06-03).[2021-02-24]. https://github.com/bojone/kg-2019-baseline.
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
