Identifying Relationship of Chinese Characters with Attention Mechanism and Convolutional Neural Network
Zhao Pengwu1,Li Zhiyi2(),Lin Xiaoqi2
1Institute of Scientific and Technical Information of China, Beijing 100038, China 2School of Economics & Management, South China Normal University, Guangzhou 510006, China
[Objective] The paper tries to identify the features and relationship of dynamic semantic information from the Chinese character entities. [Methods] First, we used the attention mechanism and improved convolution neural network model to automatically extract features from the training data of public corpus with character entity relationship. Then, we compared our model’s performance with the existing ones from the perspectives of entity relationship recognition efficiency, as well as entity relationship extraction effects and efficiency. [Results] The performance of CNN+Attention model is better than those of the SVM, LR, LSTM, BiLSTM and CNN model in prediction accuracy. Our new model is 0.92% higher in accuracy, 0.80% higher in recall and 0.86% higher in F1 value than the BiLSTM model with relatively better extraction effect. [Limitations] We need to examine our model with more sample data sets. [Conclusions] The proposed model could effectively improve the accuracy and recall of entity relationship extraction for Chinese characters.
赵鹏武, 李志义, 林小琦. 基于注意力机制和卷积神经网络的中文人物关系抽取与识别*[J]. 数据分析与知识发现, 2022, 6(8): 41-51.
Zhao Pengwu, Li Zhiyi, Lin Xiaoqi. Identifying Relationship of Chinese Characters with Attention Mechanism and Convolutional Neural Network. Data Analysis and Knowledge Discovery, 2022, 6(8): 41-51.
Bekoulis G, Deleu J, Demeester T, et al. Joint Entity Recognition and Relation Extraction as A Multi-Head Selection Problem[J]. Expert Systems with Applications, 2018, 114: 34-45.
doi: 10.1016/j.eswa.2018.07.032
[2]
Kim E K, Choi K S. Improving Distantly Supervised Relation Extraction by Knowledge Base-Driven Zero Subject Resolution[J]. IEICE Transactions on Information and Systems, 2018, E101.D(10): 2551-2558.
doi: 10.1587/transinf.2017EDL8270
(Jia Zhen, Ye Zhonglin, Yin Hongfeng, et al. Weakly Supervised Relation Extraction Based on Tri-training and Noise Filtering[J]. Journal of Chinese Information Processing, 2016, 30(4): 142-149, 158.)
(Gan Lixin, Wan Changxuan, Liu Dexi, et al. Chinese Named Entity Relation Extraction Based on Syntactic and Semantic Features[J]. Journal of Computer Research and Development, 2016, 53(2): 284-302.)
(Wu Sheng, Liu Maofu, Hu Huijun, et al. Unsupervised Extraction of Attribute-Value Entity Relation from Chinese Texts[J]. Journal of Wuhan University (Natural Science Edition), 2016, 62(6): 552-560.)
(Rong Bohui, Fu Kun, Huang Yu, et al. Relation Extraction Based on Multi-Channel Convolutional Neural Network[J]. Application Research of Computers, 2017, 34(3): 689-692.)
(Zhang Xiaobin, Chen Fucai, Huang Ruiyang. Relation Extraction Based on CNN and Bi-LSTM[J]. Chinese Journal of Network and Information Security, 2018, 4(9): 44-51.)
(Yan Haoran, Jin Xiaolong, Jia Yantao, et al. An Improved Entity Relation Extraction Algorithm——OptMultiR[J]. Journal of Chinese Information Processing, 2018, 32(9): 66-74.)
(Che Jinli, Tang Liwei, Deng Shijie, et al. Distant Supervision Chinese Relation Extraction Based on Dual Attention Mechanism[J]. Computer Engineering and Applications, 2019, 55(20): 107-113.)
doi: 10.3778/j.issn.1002-8331.1806-0438
[10]
Cui Z Y, Pan L, Liu S J. Hybrid BiLSTM-Siamese Network for Relation Extraction[C]// Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems. 2019: 1907-1909.
(Yan Yue, Huo Qirun, Li Tianhao, et al. Design and Implementation of Text Classification Based on Convolutional Neural Network with Multiple Attention Mechanisms[J]. Journal of Chinese Computer Systems, 2021, 42(2): 362-367.)
(Zhang Tingting, Li Weijiang, Li Tao. Entity Relation Extraction Based on Self-Attention Convolutional Neural Network[J]. Information Technology, 2022, 46(1): 11-15.)
(Feng Jianzhou, Song Shasha, Wang Yuanzhuo, et al. Entity Relation Extraction Based on Improved Attention Mechanism[J]. Acta Electronica Sinica, 2019, 47(8): 1692-1700.)
doi: 10.3969/j.issn.0372-2112.2019.08.012
(Zhang Lanxia, Hu Wenxin. Character Relation Extraction in Chinese Text Based on Bidirectional GRU Neural Network and Dual-attention Mechanism[J]. Computer Applications and Software, 2018, 35(11): 130-135, 189.)
(Huang Yangchen, Jia Yan, Gan Liang, et al. Multi-Factor Person Entity Relation Extraction Model Based on Distant Supervision[J]. Journal on Communications, 2018, 39(7): 103-112.)
[16]
Lv C Y, Pan D, Li Y X, et al. A Novel Chinese Entity Relationship Extraction Method Based on the Bidirectional Maximum Entropy Markov Model[J]. Complexity, 2021, 2021: 6610965.
[17]
Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition[OL]. arXiv Preprint, arXiv: 1409.1556.
[18]
Mikolov T, Chen K, Corrado G, et al. Efficient Estimation of Word Representations in Vector Space[OL]. arXiv Preprint, arXiv: 1301.3781
[19]
Lecun Y, Bottou L, Bengio Y, et al. Gradient-Based Learning Applied to Document Recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
doi: 10.1109/5.726791
[20]
Mnih V, Heess N, Graves A. Recurrent Models of Visual Attention[C]// Proceedings of the 27th Annual Conference on Neural Information Processing Systems. 2014: 2204-2212.
[21]
Chen X, Hsieh C J, Gong B. When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations[OL]. arXiv Preprint, arXiv: 2106.01548.
[22]
Elsayed G F, Kornblith S, Le Q V. Saccader: Improving Accuracy of Hard Attention Models for Vision[OL]. arXiv Preprint, arXiv: 1908.07644.
[23]
Luong T, Pham H, Manning C D. Effective Approaches to Attention-based Neural Machine Translation[C]// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015: 2412-2421.
[24]
Chinchor N, Sundheim B. MUC-5 Evaluation Metrics[C]// Proceedings of the 5th Conference on Message Understanding. 1993: 69-78.