Please wait a minute...
Advanced Search
数据分析与知识发现  2022, Vol. 6 Issue (8): 41-51     https://doi.org/10.11925/infotech.2096-3467.2021.1079
  研究论文 本期目录 | 过刊浏览 | 高级检索 |
基于注意力机制和卷积神经网络的中文人物关系抽取与识别*
赵鹏武1,李志义2(),林小琦2
1中国科学技术信息研究所 北京 100038
2华南师范大学经济与管理学院 广州 510006
Identifying Relationship of Chinese Characters with Attention Mechanism and Convolutional Neural Network
Zhao Pengwu1,Li Zhiyi2(),Lin Xiaoqi2
1Institute of Scientific and Technical Information of China, Beijing 100038, China
2School of Economics & Management, South China Normal University, Guangzhou 510006, China
全文: PDF (1321 KB)   HTML ( 31
输出: BibTeX | EndNote (RIS)      
摘要 

【目的】 研究中文人物实体关系中动态语义信息的特征抽取和中文人物关系识别。【方法】 采用公开的人物实体关系语料集,使用注意力机制+改进的卷积神经网络模型(CNN+Attention)从训练数据中自动提取特征,从不同模型实体关系识别效率、不同关系标签实体关系抽取效果以及不同向量训练集实体关系抽取效率等多维度进行对比和验证。【结果】 在中文人物关系抽取任务上,CNN+Attention模型的预测准确率和全局性能均优于SVM、LR、LSTM、BiLSTM以及CNN模型,并比抽取效果相对较优的BiLSTM模型准确率提高0.92个百分点,召回率提高0.86个百分点,F1值提高0.80个百分点。【局限】 仅使用单一的样本数据来源,未拓展多种数据来源渠道,样本数据集范围不够广。【结论】 基于注意力机制的卷积神经网络,在中文人物关系抽取任务中能够有效地提升实体关系抽取的准确率和召回率。

服务
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章
赵鹏武
李志义
林小琦
关键词 卷积神经网络注意力机制中文人物关系抽取关系识别    
Abstract

[Objective] The paper tries to identify the features and relationship of dynamic semantic information from the Chinese character entities. [Methods] First, we used the attention mechanism and improved convolution neural network model to automatically extract features from the training data of public corpus with character entity relationship. Then, we compared our model’s performance with the existing ones from the perspectives of entity relationship recognition efficiency, as well as entity relationship extraction effects and efficiency. [Results] The performance of CNN+Attention model is better than those of the SVM, LR, LSTM, BiLSTM and CNN model in prediction accuracy. Our new model is 0.92% higher in accuracy, 0.80% higher in recall and 0.86% higher in F1 value than the BiLSTM model with relatively better extraction effect. [Limitations] We need to examine our model with more sample data sets. [Conclusions] The proposed model could effectively improve the accuracy and recall of entity relationship extraction for Chinese characters.

Key wordsConvolutional Neural Network    Attention Mechanism    Chinese Character    Relationship Extraction    Relationship Recognition
收稿日期: 2021-09-22      出版日期: 2022-09-23
ZTFLH:  N99  
基金资助:*国家社会科学基金项目的研究成果之一(17BTQ062)
通讯作者: 李志义,ORCID: 0000-0001-6407-2554     E-mail: leeds@scnu.edu.cn
引用本文:   
赵鹏武, 李志义, 林小琦. 基于注意力机制和卷积神经网络的中文人物关系抽取与识别*[J]. 数据分析与知识发现, 2022, 6(8): 41-51.
Zhao Pengwu, Li Zhiyi, Lin Xiaoqi. Identifying Relationship of Chinese Characters with Attention Mechanism and Convolutional Neural Network. Data Analysis and Knowledge Discovery, 2022, 6(8): 41-51.
链接本文:  
https://manu44.magtech.com.cn/Jwk_infotech_wk3/CN/10.11925/infotech.2096-3467.2021.1079      或      https://manu44.magtech.com.cn/Jwk_infotech_wk3/CN/Y2022/V6/I8/41
Fig.1  CNN+Attention模型结构
关系 ID
Unknown 0
父母 1
合作 2
师生 3
投资 4
对手 5
情侣 6
雇佣 7
朋友 8
亲戚 9
同学 10
上下级 11
Table 1  关系和对应ID明细表
Fig.2  研究方案流程
属性 属性值
Keras版本 2.2.4
TensorFlow版本 1.10
GPU NVIDIA GeForce GTX 1080Ti
CPU AMD 2950X@3.50GHz
内存 32GB
操作环境 Linux,Ubuntu16.05
Table 2  实验环境
参数名称 参数值
Embedding维度 200
Epochs 250
Batch size 256
Dropout rate 0.3
Learning rate 0.001
Table 3  实体关系抽取实验参数设置
预测 实际有关系 实际无关系
预测有关系 True Positive(TP False Positive(FP
预测无关系 False Negative(FN True Negative(TN
Table 4  P-R混淆矩阵
模型 P R F1
SVM 0.901 8 0.874 3 0.887 8
LR 0.783 3 0.764 2 0.773 6
LSTM 0.938 4 0.935 3 0.936 7
BiLSTM 0.940 8 0.932 1 0.936 3
CNN 0.926 4 0.926 4 0.864 1
CNN+Attention 0.950 0 0.940 1 0.944 9
Table 5  机器学习模型实验结果
关系 P R F1
父母 0.941 4 0.946 3 0.944 1
合作 0.957 6 0.960 2 0.959 0
师生 0.943 8 0.923 5 0.933 6
投资 0.953 9 0.933 6 0.943 6
对手 0.952 8 0.921 4 0.936 8
情侣 0.942 0 0.936 3 0.939 1
雇佣 0.965 0 0.915 3 0.939 5
朋友 0.949 2 0.880 9 0.913 8
亲戚 0.911 2 0.868 4 0.889 3
同学 0.973 3 0.921 9 0.929 7
上下级 0.971 1 0.950 5 0.960 7
Table 6  CNN+Attention实体关系抽取效果
Fig.3  CNN+Attention模型准确率和损失率变化
向量 P R F1
CNN+Attention(随机初始化向量) 0.876 1 0.848 3 0.861 9
CNN+Attention(经训练的向量) 0.917 6 0.887 5 0.902 1
Table 7  随机化向量与训练向量关系抽取效果对比
Fig.4  实体关系抽取PR图
Fig.5  CNN+Attention模型平均准确率
[1] Bekoulis G, Deleu J, Demeester T, et al. Joint Entity Recognition and Relation Extraction as A Multi-Head Selection Problem[J]. Expert Systems with Applications, 2018, 114: 34-45.
doi: 10.1016/j.eswa.2018.07.032
[2] Kim E K, Choi K S. Improving Distantly Supervised Relation Extraction by Knowledge Base-Driven Zero Subject Resolution[J]. IEICE Transactions on Information and Systems, 2018, E101.D(10): 2551-2558.
doi: 10.1587/transinf.2017EDL8270
[3] 贾真, 冶忠林, 尹红风, 等. 基于Tri-training与噪声过滤的弱监督关系抽取[J]. 中文信息学报, 2016, 30(4): 142-149, 158.
[3] (Jia Zhen, Ye Zhonglin, Yin Hongfeng, et al. Weakly Supervised Relation Extraction Based on Tri-training and Noise Filtering[J]. Journal of Chinese Information Processing, 2016, 30(4): 142-149, 158.)
[4] 甘丽新, 万常选, 刘德喜, 等. 基于句法语义特征的中文实体关系抽取[J]. 计算机研究与发展, 2016, 53(2): 284-302.
[4] (Gan Lixin, Wan Changxuan, Liu Dexi, et al. Chinese Named Entity Relation Extraction Based on Syntactic and Semantic Features[J]. Journal of Computer Research and Development, 2016, 53(2): 284-302.)
[5] 吴胜, 刘茂福, 胡慧君, 等. 中文文本中实体数值型关系无监督抽取方法[J]. 武汉大学学报(理学版), 2016, 62(6): 552-560.
[5] (Wu Sheng, Liu Maofu, Hu Huijun, et al. Unsupervised Extraction of Attribute-Value Entity Relation from Chinese Texts[J]. Journal of Wuhan University (Natural Science Edition), 2016, 62(6): 552-560.)
[6] 肜博辉, 付琨, 黄宇, 等. 基于多通道卷积神经网的实体关系抽取[J]. 计算机应用研究, 2017, 34(3): 689-692.
[6] (Rong Bohui, Fu Kun, Huang Yu, et al. Relation Extraction Based on Multi-Channel Convolutional Neural Network[J]. Application Research of Computers, 2017, 34(3): 689-692.)
[7] 张晓斌, 陈福才, 黄瑞阳. 基于CNN和双向LSTM融合的实体关系抽取[J]. 网络与信息安全学报, 2018, 4(9): 44-51.
[7] (Zhang Xiaobin, Chen Fucai, Huang Ruiyang. Relation Extraction Based on CNN and Bi-LSTM[J]. Chinese Journal of Network and Information Security, 2018, 4(9): 44-51.)
[8] 延浩然, 靳小龙, 贾岩涛, 等. 一种改进的实体关系抽取算法——OptMultiR[J]. 中文信息学报, 2018, 32(9): 66-74.
[8] (Yan Haoran, Jin Xiaolong, Jia Yantao, et al. An Improved Entity Relation Extraction Algorithm——OptMultiR[J]. Journal of Chinese Information Processing, 2018, 32(9): 66-74.)
[9] 车金立, 唐力伟, 邓士杰, 等. 基于双重注意力机制的远程监督中文关系抽取[J]. 计算机工程与应用, 2019, 55(20): 107-113.
doi: 10.3778/j.issn.1002-8331.1806-0438
[9] (Che Jinli, Tang Liwei, Deng Shijie, et al. Distant Supervision Chinese Relation Extraction Based on Dual Attention Mechanism[J]. Computer Engineering and Applications, 2019, 55(20): 107-113.)
doi: 10.3778/j.issn.1002-8331.1806-0438
[10] Cui Z Y, Pan L, Liu S J. Hybrid BiLSTM-Siamese Network for Relation Extraction[C]// Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems. 2019: 1907-1909.
[11] 闫跃, 霍其润, 李天昊, 等. 融合多重注意力机制的卷积神经网络文本分类设计与实现[J]. 小型微型计算机系统, 2021, 42(2): 362-367.
[11] (Yan Yue, Huo Qirun, Li Tianhao, et al. Design and Implementation of Text Classification Based on Convolutional Neural Network with Multiple Attention Mechanisms[J]. Journal of Chinese Computer Systems, 2021, 42(2): 362-367.)
[12] 张婷婷, 李卫疆, 李涛. 基于自注意力卷积神经网络的实体关系抽取[J]. 信息技术, 2022, 46(1): 11-15.
[12] (Zhang Tingting, Li Weijiang, Li Tao. Entity Relation Extraction Based on Self-Attention Convolutional Neural Network[J]. Information Technology, 2022, 46(1): 11-15.)
[13] 冯建周, 宋沙沙, 王元卓, 等. 基于改进注意力机制的实体关系抽取方法[J]. 电子学报, 2019, 47(8): 1692-1700.
doi: 10.3969/j.issn.0372-2112.2019.08.012
[13] (Feng Jianzhou, Song Shasha, Wang Yuanzhuo, et al. Entity Relation Extraction Based on Improved Attention Mechanism[J]. Acta Electronica Sinica, 2019, 47(8): 1692-1700.)
doi: 10.3969/j.issn.0372-2112.2019.08.012
[14] 张兰霞, 胡文心. 基于双向GRU神经网络和双层注意力机制的中文文本中人物关系抽取研究[J]. 计算机应用与软件, 2018, 35(11): 130-135, 189.
[14] (Zhang Lanxia, Hu Wenxin. Character Relation Extraction in Chinese Text Based on Bidirectional GRU Neural Network and Dual-attention Mechanism[J]. Computer Applications and Software, 2018, 35(11): 130-135, 189.)
[15] 黄杨琛, 贾焰, 甘亮, 等. 基于远程监督的多因子人物关系抽取模型[J]. 通信学报, 2018, 39(7): 103-112.
[15] (Huang Yangchen, Jia Yan, Gan Liang, et al. Multi-Factor Person Entity Relation Extraction Model Based on Distant Supervision[J]. Journal on Communications, 2018, 39(7): 103-112.)
[16] Lv C Y, Pan D, Li Y X, et al. A Novel Chinese Entity Relationship Extraction Method Based on the Bidirectional Maximum Entropy Markov Model[J]. Complexity, 2021, 2021: 6610965.
[17] Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition[OL]. arXiv Preprint, arXiv: 1409.1556.
[18] Mikolov T, Chen K, Corrado G, et al. Efficient Estimation of Word Representations in Vector Space[OL]. arXiv Preprint, arXiv: 1301.3781
[19] Lecun Y, Bottou L, Bengio Y, et al. Gradient-Based Learning Applied to Document Recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
doi: 10.1109/5.726791
[20] Mnih V, Heess N, Graves A. Recurrent Models of Visual Attention[C]// Proceedings of the 27th Annual Conference on Neural Information Processing Systems. 2014: 2204-2212.
[21] Chen X, Hsieh C J, Gong B. When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations[OL]. arXiv Preprint, arXiv: 2106.01548.
[22] Elsayed G F, Kornblith S, Le Q V. Saccader: Improving Accuracy of Hard Attention Models for Vision[OL]. arXiv Preprint, arXiv: 1908.07644.
[23] Luong T, Pham H, Manning C D. Effective Approaches to Attention-based Neural Machine Translation[C]// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 2015: 2412-2421.
[24] Chinchor N, Sundheim B. MUC-5 Evaluation Metrics[C]// Proceedings of the 5th Conference on Message Understanding. 1993: 69-78.
[1] 叶瀚,孙海春,李欣,焦凯楠. 融合注意力机制与句向量压缩的长文本分类模型[J]. 数据分析与知识发现, 2022, 6(6): 84-94.
[2] 景慎旗, 赵又霖. 基于医学领域知识和远程监督的医学实体关系抽取研究*[J]. 数据分析与知识发现, 2022, 6(6): 105-114.
[3] 张若琦, 申建芳, 陈平华. 结合GNN、Bi-GRU及注意力机制的会话序列推荐*[J]. 数据分析与知识发现, 2022, 6(6): 46-54.
[4] 郭樊容, 黄孝喜, 王荣波, 谌志群, 胡创, 谢一敏, 司博宇. 基于Transformer和图卷积神经网络的隐喻识别*[J]. 数据分析与知识发现, 2022, 6(4): 120-129.
[5] 郭航程, 何彦青, 兰天, 吴振峰, 董诚. 基于Paragraph-BERT-CRF的科技论文摘要语步功能信息识别方法研究*[J]. 数据分析与知识发现, 2022, 6(2/3): 298-307.
[6] 徐月梅, 樊祖薇, 曹晗. 基于标签嵌入注意力机制的多任务文本分类模型*[J]. 数据分析与知识发现, 2022, 6(2/3): 105-116.
[7] 王永生, 王昊, 虞为, 周泽聿. 融合结构和内容的方志文本人物关系抽取方法*[J]. 数据分析与知识发现, 2022, 6(2/3): 318-328.
[8] 谢星雨, 余本功. 基于MFFMB的电商评论文本分类研究*[J]. 数据分析与知识发现, 2022, 6(1): 101-112.
[9] 范少萍,赵雨宣,安新颖,吴清强. 基于卷积神经网络的医学实体关系分类模型研究*[J]. 数据分析与知识发现, 2021, 5(9): 75-84.
[10] 王一钒,李博,史话,苗威,姜斌. 古汉语实体关系联合抽取的标注方法*[J]. 数据分析与知识发现, 2021, 5(9): 63-74.
[11] 范涛,王昊,吴鹏. 基于图卷积神经网络和依存句法分析的网民负面情感分析研究*[J]. 数据分析与知识发现, 2021, 5(9): 97-106.
[12] 马江微, 吕学强, 游新冬, 肖刚, 韩君妹. 融合BERT与关系位置特征的军事领域关系抽取方法*[J]. 数据分析与知识发现, 2021, 5(8): 1-12.
[13] 杨晗迅, 周德群, 马静, 罗永聪. 基于不确定性损失函数和任务层级注意力机制的多任务谣言检测研究*[J]. 数据分析与知识发现, 2021, 5(7): 101-110.
[14] 谢豪,毛进,李纲. 基于多层语义融合的图文信息情感分类研究*[J]. 数据分析与知识发现, 2021, 5(6): 103-114.
[15] 尹鹏博,潘伟民,张海军,陈德刚. 基于BERT-BiGA模型的标题党新闻识别研究*[J]. 数据分析与知识发现, 2021, 5(6): 126-134.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
版权所有 © 2015 《数据分析与知识发现》编辑部
地址:北京市海淀区中关村北四环西路33号 邮编:100190
电话/传真:(010)82626611-6626,82624938
E-mail:jishu@mail.las.ac.cn