Please wait a minute...
Advanced Search
数据分析与知识发现  2020, Vol. 4 Issue (4): 91-99    DOI: 10.11925/infotech.2096-3467.2019.0828
  研究论文 本期目录 | 过刊浏览 | 高级检索 |
基于词嵌入融合和循环神经网络的中英文隐喻识别*
苏传东,黄孝喜(),王荣波,谌志群,毛君钰,朱嘉莹,潘宇豪
杭州电子科技大学认知与智能计算研究所 杭州 310018
Identifying Chinese / English Metaphors with Word Embedding and Recurrent Neural Network
Su Chuandong,Huang Xiaoxi(),Wang Rongbo,Chen Zhiqun,Mao Junyu,Zhu Jiaying,Pan Yuhao
Institute of Cognitive and Intelligent Computing, Hangzhou Dianzi University, Hangzhou 310018, China
全文: PDF(825 KB)   HTML ( 3
输出: BibTeX | EndNote (RIS)      
摘要 

【目的】 针对自然语言中普遍存在的隐喻现象,提出一种基于词向量融合和循环神经网络(RNN)的中英文隐喻识别方法。【方法】 通过本文提出的词嵌入融合算法将文本映射到词向量空间作为神经网络的输入,以RNN作为编码器,注意力机制和池化技术作为特征提取器,最后利用Softmax计算文本为隐喻的概率。【结果】 基于词嵌入融合的隐喻识别方法的准确率和F1值比基于普通词嵌入的方法在英文隐喻识别任务上可以提高11.8%和6.3%,在中文隐喻识别任务上可以提高8.9%和7.8%。【局限】 由于长距离依存问题,本文方法在句式复杂的长文本上隐喻识别效果不稳定。【结论】 基于词嵌入融合和RNN的模型在隐喻识别问题上表现非常好,说明词嵌入融合可以提高神经网络对隐喻的识别能力。

服务
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章
苏传东
黄孝喜
王荣波
谌志群
毛君钰
朱嘉莹
潘宇豪
关键词 隐喻识别深度学习词嵌入循环神经网络    
Abstract

[Objective] This paper proposes a method to recognize Chinese and English metaphors with word vector combination and recurrent neural network (RNN), aiming to identify the ubiquitous metaphors from natural languages. [Methods] First, we mapped texts to the word vectors as inputs of the neural network with the help of word-embedding combination algorithm. Then, we used the RNN as encoder, and took the attention mechanism and the pooling technique as feature extractor. Finally, we utilized Softmax to calculate the probability of the text was a metaphor. [Results] The accuracy and F1 of the proposed method with English texts improved by 11.8% and 6.3%, compared with traditional method based on vanilla word embedding. For Chinese tasks, the accuracy and F1 of the proposed method also improved by 8.9% and 7.8%. [Limitations] Due to the long-distance dependence issue, our method could not effectively recognize metaphors in long texts with complex sentences. [Conclusions] The proposed model signifcantly improves the neural network’s ability to recognize metaphors.

Key wordsMetaphor Recognition    Deep Learning    Word Embedding    Recurrent Neural Network
收稿日期: 2019-07-12     
中图分类号:  TP391  
基金资助:*本文系教育部人文社会科学研究规划基金项目“融合深度神经网络模型的汉语隐喻计算研究”(18YJA740016);国家社会科学基金重大规划项目“汉语隐喻的逻辑表征与认知计算”的研究成果之一(18ZDA290)
通讯作者: 黄孝喜     E-mail: huangxx@hdu.edu.cn
引用本文:   
苏传东,黄孝喜,王荣波,谌志群,毛君钰,朱嘉莹,潘宇豪. 基于词嵌入融合和循环神经网络的中英文隐喻识别*[J]. 数据分析与知识发现, 2020, 4(4): 91-99.
Su Chuandong,Huang Xiaoxi,Wang Rongbo,Chen Zhiqun,Mao Junyu,Zhu Jiaying,Pan Yuhao. Identifying Chinese / English Metaphors with Word Embedding and Recurrent Neural Network. Data Analysis and Knowledge Discovery, DOI:10.11925/infotech.2096-3467.2019.0828.
链接本文:  
http://manu44.magtech.com.cn/Jwk_infotech_wk3/CN/10.11925/infotech.2096-3467.2019.0828
图1  词嵌入向量运算示意
图2  通过词嵌入解释隐喻示意
图3  词嵌入融合示意
图4  基于循环神经网络的隐喻识别器架构
模型 词嵌入方式 数据集 准确率 精确率 召回率 F1值
MR G TroFi 65.3 56.1 88.5 67.7
MR F TroFi 62.0 58.5 83.6 67.3
MR P TroFi 63.6 54.0 85.9 66.3
EC-MR M1(G,P) TroFi 73.8 66.3 75.6 70.7
EC-MR M1(G,F,P) TroFi 71.1 63.3 73.1 67.9
EC-MR M2(G,P) TroFi 66.8 56.9 84.6 68.0
EC-MR M2(G,F,P) TroFi 64.2 54.3 88.6 67.3
SEQ - TroFi 73.7 68.7 76.4 72.0
MR G AN 84.2 76.0 95.0 84.3
MR F AN 80.9 79.5 77.5 78.5
MR P AN 83.1 76.6 90.0 82.8
EC-RNN M1(G,P) AN 83.1 77.8 87.5 82.4
EC-RNN M1(G,F,P) AN 82.0 74.0 92.5 82.2
EC-RNN M2(G,P) AN 84.3 75.0 97.5 84.8
EC-RNN M2(G,F,P) AN 86.5 86.8 82.5 84.6
SSN - AN 82.9 90.3 73.8 81.1
表1  隐喻识别器在英文隐喻识别任务上的表现(%)
模型 词嵌入方式 数据集 准确率 精确率 召回率 F1
MR B TroFi_CN 58.3 50.0 96.1 65.8
MR S TroFi_CN 59.9 51.6 85.9 64.1
MR W TroFi_CN 58.8 50.3 91.4 66.4
EC-MR M1(B,S) TroFi_CN 61.5 52.1 94.9 67.3
EC-MR M1(B,S,W) TroFi_CN 61.0 51.7 96.2 67.2
EC-MR M2(B,S) TroFi_CN 59.9 50.1 93.6 66.1
EC-MR M2(B,S,W) TroFi_CN 59.4 50.7 92.3 65.5
MR B AN_CN 84.3 82.5 82.4 82.5
MR S AN_CN 77.5 71.7 82.4 76.7
MR W AN_CN 84.3 82.6 84.5 83.3
EC-MR M1(B,W) AN_CN 85.4 84.6 82.5 83.5
EC-MR M1(B,S,W) AN_CN 85.4 86.5 80.0 83.1
EC-MR M2(B,W) AN_CN 85.4 82.9 85.0 84.0
EC-MR M2(B,S,W) AN_CN 86.4 86.8 82.6 84.5
表2  隐喻识别器在中文隐喻识别任务上的表现(%)
[1] Lakoff G, Johnson M . Conceptual Metaphor in Everyday Language[J]. The Journal of Philosophy, 1980,77(8):453-486.
doi: 10.2307/2025464
[2] Tsvetkov Y, Boytsov L, Gershman A, et al. Metaphor Detection with Cross-Lingual Model Transfer[C]// Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. 2014,1:248-258.
[3] McArthur T, Lam-McArthur J, Fontaine L . Oxford Companion to the English Language[M]. Oxford University Press, 2018.
[4] Shutova E . Design and Evaluation of Metaphor Processing Systems[J]. Computational Linguistics, 2015,41(4):579-623.
doi: 10.1162/COLI_a_00233
[5] Mao R, Lin C, Guerin F. Word Embedding and WordNet Based Metaphor Identification and Interpretation[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018: 1222-1231.
[6] Wilks Y . A Preferential , Pattern-seeking, Semantics for Natural Language Inference[A]// Ahmad K, Brewster C, Stevenson M. Words and Intelligence I[M]. Springer, 2007: 83-102.
[7] Fass D . Met*: A Method for Discriminating Metonymy and Metaphor by Computer[J]. Computational Linguistics, 1991,17(1):49-90.
[8] Neuman Y, Assaf D, Cohen Y , et al. Metaphor Identification in Large Texts Corpora[J]. PLoS One, 2013,8(4):e62343.
doi: 10.1371/journal.pone.0062343
[9] Shutova E, Sun L, Korgonen A. Metaphor Identification Using Verb and Nouns Clustering[C]// Proceedings of the 23rd International Conference on Computational Linguistics. 2010: 1002-1010.
[10] Hovy D, Srivastava S, Jauhar S K, et al. Identifying Metaphorical Word Use with Tree Kernels[C]// Proceedings of the 1st Workshop on Metaphor in NLP. 2013: 52-57.
[11] Rai S, Chakraverty S, Tayal D K. Supervised Metaphor Detection Using Conditional Random Fields[C]// Proceedings of the 4th Workshop on Metaphor in NLP. 2016: 18-27.
[12] Kalchbrenner N, Grefenstette E, Blunsom P . A Convolutional Neural Network for Modelling Sentences[OL]. arXiv Preprint, arXiv: 1404. 2188.
[13] Graves A, Schmidhuber J . Framewise Phoneme Classification with Bidirectional LSTM and Other Neural Network Architectures[J]. Neural Networks, 2005,18(5-6):602-610.
doi: 10.1016/j.neunet.2005.06.042
[14] Do Dinh E L, Gurevych I. Token-Level Metaphor Detection Using Neural Networks[C]// Proceedings of the 4th Workshop on Metaphor in NLP. 2016: 28-33.
[15] Bizzoni Y, Chatzikyriakidis S, Ghanimifard M. “Deep” Learning: Detecting Metaphoricity in Adjective-Noun Pairs[C]// Proceedings of the 2017 Workshop on Stylistic Variation. 2017: 43-52.
[16] Rei M, Bulat L, Kiela D , et al. Grasping the Finer Point: A Supervised Similarity Network for Metaphor Detection[OL]. arXiv Preprint, arXiv: 1709. 00575.
[17] 王治敏, 王厚峰, 俞士汶 . 基于机器学习方法的汉语名词隐喻识别[J]. 高技术通讯 , 2006,17(6):575-580.
( Wang Zhimin, Wang Houfeng, Yu Shiwen . Chinese Nominal Metaphor Recognition Based on Machine Learning[J]. Chinese High Technology Letters, 2006,17(6):575-580.)
[18] 李斌, 于丽丽, 石民 , 等. “像”的明喻计算[J]. 中文信息学报, 2008,22(6):27-32.
( Li Bin, Yu Lili, Shi Min , et al. Computation of Chinese Simile with “Xiang”[J]. Journal of Chinese Information Processing, 2008,22(6):27-32.)
[19] 黄孝喜 . 隐喻机器理解的若干关键问题研究[D]. 杭州: 浙江大学, 2009.
( Huang Xiaoxi . Research on Some Key Issues of Metaphor Computation[D]. Hangzhou: Zhejiang University, 2009.)
[20] 黄孝喜, 李晗雨, 王荣波 , 等. 基于卷积神经网络与 SVM 分类器的隐喻识别[J]. 数据分析与知识发现, 2018,2(10):77-83.
( Huang Xiaoxi, Li Hanyu, Wang Rongbo , et al. Recognizing Metaphor with Convolution Neural Network and SVM[J]. Data Analysis and Knowledge Discovery, 2018,2(10):77-83.)
[21] Mikolov T, Chen K, Corrado G , et al. Efficient Estimation of Word Representations in Vector Space[OL]. arXiv Preprint, arXiv: 1301. 3781.
[22] Mikolov T, Sutskever I, Chen K, et al. Distributed Representations of Words and Phrases and Their Compositionality[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013: 3111-3119.
[23] Coates J, Bollegala D . Frustratingly Easy Meta-Embedding—Computing Meta-Embeddings by Averaging Source Word Embeddings[OL]. arXiv Preprint, arXiv: 1804. 05262.
[24] Graves A, Fernández S, Schmidhuber J. Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition[C]// Proceedings of the 15th International Conference on Artificial Neural Networks: Formal Models and Their Applications. Springer, 2005: 799-804.
[25] Bahdanau D, Cho K, Bengio Y . Neural Machine Translation by Jointly Learning to Align and Translate[OL]. arXiv Preprint, arXiv: 1409. 0473.
[26] Birke J, Sarkar A. A Clustering Approach for Nearly Unsupervised Recognition of Nonliteral Language[C]// Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics. 2006.
[27] Pennington J, Socher R, Manning C. GloVe: Global Vectors for Word Representation[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014: 1532-1543.
[28] Joulin A, Grave E, Bojanowski P , et al. Bag of Tricks for Efficient Text Classification[OL]. arXiv Preprint, arXiv: 1607. 01759.
[29] Wieting J, Bansal M, Gimpel K , et al. From Paraphrase Database to Compositional Paraphrase Model and Back[J]. Transactions of the Association for Computational Linguistics, 2015,3:345-358.
[30] Gao G, Choi E, Choi Y , et al. Neural Metaphor Detection in Context[OL]. arXiv Preprint, arXiv: 1808. 09653.
[31] Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017: 5998-6008.
[32] Radford A, Narasimhan K, Salimans T , et al. Improving Language Understanding by Generative Pre-training[OL]. https://www.cs.ubc.ca/~amuham01/LING530/papers/radford2018improving.pdf.
[1] 王末,崔运鹏,陈丽,李欢. 基于深度学习的学术论文语步结构分类方法研究*[J]. 数据分析与知识发现, 2020, 4(6): 60-68.
[2] 焦启航,乐小虬. 对比关系句子生成方法研究[J]. 数据分析与知识发现, 2020, 4(6): 43-50.
[3] 邓思艺,乐小虬. 基于动态语义注意力的指代消解方法[J]. 数据分析与知识发现, 2020, 4(5): 46-53.
[4] 余传明,原赛,朱星宇,林虹君,张普亮,安璐. 基于深度学习的热点事件主题表示研究*[J]. 数据分析与知识发现, 2020, 4(4): 1-14.
[5] 张冬瑜,崔紫娟,李映夏,张伟,林鸿飞. 基于Transformer和BERT的名词隐喻识别*[J]. 数据分析与知识发现, 2020, 4(4): 100-108.
[6] 刘彤,倪维健,孙宇健,曾庆田. 基于深度迁移学习的业务流程实例剩余执行时间预测方法*[J]. 数据分析与知识发现, 2020, 4(2/3): 134-142.
[7] 宰新宇,田学东. 基于公式描述结构和词嵌入的科技文档检索方法*[J]. 数据分析与知识发现, 2020, 4(1): 131-138.
[8] 余传明,李浩男,王曼怡,黄婷婷,安璐. 基于深度学习的知识表示研究:网络视角*[J]. 数据分析与知识发现, 2020, 4(1): 63-75.
[9] 曾庆田,胡晓慧,李超. 融合主题词嵌入和网络结构分析的主题关键词提取方法 *[J]. 数据分析与知识发现, 2019, 3(7): 52-60.
[10] 张梦吉,杜婉钰,郑楠. 引入新闻短文本的个股走势预测模型[J]. 数据分析与知识发现, 2019, 3(5): 11-18.
[11] 裴晶晶,乐小虬. 篇章级并列关系文本块识别方法研究[J]. 数据分析与知识发现, 2019, 3(5): 51-56.
[12] 张智雄,刘欢,丁良萍,吴朋民,于改红. 不同深度学习模型的科技论文摘要语步识别效果对比研究 *[J]. 数据分析与知识发现, 2019, 3(12): 1-9.
[13] 余丽,钱力,付常雷,赵华茗. 基于深度学习的文本中细粒度知识元抽取方法研究*[J]. 数据分析与知识发现, 2019, 3(1): 38-45.
[14] 付常雷,钱力,张华平,赵华茗,谢靖. 基于深度学习的创新主题智能挖掘算法研究*[J]. 数据分析与知识发现, 2019, 3(1): 46-54.
[15] 余本功,张培行,许庆堂. 基于F-BiGRU情感分析的产品选择方法*[J]. 数据分析与知识发现, 2018, 2(9): 22-30.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
版权所有 © 2015 《数据分析与知识发现》编辑部
地址:北京市海淀区中关村北四环西路33号 邮编:100190
电话/传真:(010)82626611-6626,82624938
E-mail:jishu@mail.las.ac.cn