Please wait a minute...
Data Analysis and Knowledge Discovery  2018, Vol. 2 Issue (12): 23-32    DOI: 10.11925/infotech.2096-3467.2018.0583
Current Issue | Archive | Adv Search |
Comprehending Texts and Answering Questions Based on Hierarchical Interactive Network
Cheng Yong1(), Xu Dekuan1, Lv Xueqiang2
1School of Chinese Language and Literature, Ludong University, Yantai 264025, China
2School of Computer Science, Beijing University of Information Technology, Beijing 100192, China
Download: PDF (1865 KB)   HTML ( 4
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper aims to help computer answer questions accurately based on text comprehension. [Methods] First, we proposed a neural network model based on hirrarchical interaction mechanism. We introduced various human thinking mechanism to build this model, which contained hierarchical processing, content filtering and multi-dimensional attention. Then, we ran the proposed model with dataset from Chinese Machine Reading Comprehension (CMRC) 2017. [Results] The precision of the proposed method on test-set was 0.78, which was better than the best result of other published models. [Limitations] There was no further optimization for the potential answers. [Conclusions] The proposed hierarchical interactive network improves machine’s ability to answer questions based on text comprehension.

Key wordsHirarchical Interactive Network      Machine Comprehension      Automatic Question Answering     
Received: 24 May 2018      Published: 16 January 2019
ZTFLH:  G353  

Cite this article:

Cheng Yong,Xu Dekuan,Lv Xueqiang. Comprehending Texts and Answering Questions Based on Hierarchical Interactive Network. Data Analysis and Knowledge Discovery, 2018, 2(12): 23-32.

URL:

http://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2018.0583     OR     http://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2018/V2/I12/23

比较项
中央处理器 Intel Xeon CPU8 Cores
内存 DDR4 64GB
显卡 Titan Xp 12GB
单轮平均迭代时间 2小时 2分钟
总训练时间 30小时 30分钟
超参数 性能比较
词向量维度 维度 64 128 192 256
准确率 0.759 0.762 0.757 0.756
状态向量
维度
维度 64 128 192 256
准确率 0.741 0.762 0.761 0.762
神经元
保存率
保存率 0.5 0.6 0.7 0.8
准确率 0.751 0.762 0.757 0.755
网络结构 准确率(校验集) 准确率(测试集)
All information 0.762 0.775
-query_attention 0.749 0.759
-doc_attention 0.754 0.766
-add 0.759 0.770
-multi 0.759 0.769
对比方法 准确率(校验集) 准确率(测试集)
基线
模型
Random Guess 0.017 0.017
Top Frequency 0.107 0.087
As Reader[6] 0.698 0.713
GA Reader[8] 0.748 0.751
CMRC
评测方法
Top 1 0.761 0.777
Top 2 0.772 0.775
Top 3 0.779 0.774
Our Model 0.763 0.780
[1] 郭利敏. 基于卷积神经网络的文献自动分类研究[J]. 图书与情报, 2017(6): 96-103.
doi: 10.11968/tsyqb.1003-6938.2017119
[1] (Guo Limin.Study of Automatic Classification of Literature Based on Convolution Neural Network[J]. Library and Information, 2017(6): 96-103.)
doi: 10.11968/tsyqb.1003-6938.2017119
[2] 李慧宗, 胡学钢, 杨恒宇,等. 基于LDA的社会化标签综合聚类方法[J]. 情报学报, 2015, 34(2): 146-155.
doi: 10.3772/j.issn.1000-0135.2015.002.004
[2] (Li Huizong, Hu Xuegang, Yang Hengyu, et al.A Comprehensive Clustering Method of Social Tags Based on LDA[J]. Journal of the China Society for Scientific and Technical Information, 2015, 34(2): 146-155.)
doi: 10.3772/j.issn.1000-0135.2015.002.004
[3] 徐彤阳, 尹凯. 大数据背景下微博语义检索[J]. 情报杂志, 2017, 36(12): 173-179.
[3] (Xu Tongyang, Yin Kai.Semantic Retrieval of Microblogging in the Background of Large Data[J]. Journal of Intelligence, 2017, 36(12): 173-179.)
[4] 张志昌. 开放域阅读理解关键技术研究[D]. 哈尔滨: 哈尔滨工业大学, 2010.
[4] (Zhang Zhichang.Key Technologies of Reading Comprehension for Open-Domain[D]. Harbin: Harbin Institute of Technology, 2010.)
[5] Hermann K M, Kočiský T, Grefenstette E, et al.Teaching Machines to Read and Comprehend[C]// Proceedings of the 28th International Conference on Neural Information Processing Systems. 2015: 1693-1701.
[6] Kadlec R, Schmid M, Bajgar O, et al.Text Understanding with the Attention Sum Reader Network[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016: 908-918.
[7] Chen D, Bolton J, Manning C D.A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016: 2358-2367.
[8] Dhingra B, Liu H, Yang Z, et al.Gated-Attention Readers for Text Comprehension[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017: 1832-1846.
[9] Sordoni A, Bachman P, Trischler A, et al.Iterative Alternating Neural Attention for Machine Reading[OL]. arXiv Preprint, arXiv: 1606.02245.
[10] Cui Y, Chen Z, Wei S, et al.Attention-over-Attention Neural Networks for Reading Comprehension[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017: 593-602.
[11] Rajpurkar P, Zhang J, Lopyrev K, et al.SQuAD: 100,000+ Questions for Machine Comprehension of Text[C]// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2016: 2383-2392.
[12] Wang W, Yang N, Wei F, et al.Gated Self-Matching Networks for Reading Comprehension and Question Answering[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017: 189-198.
[13] Seo M, Kembhavi A, Farhadi A, et al.Bidirectional Attention Flow for Machine Comprehension[OL]. arXiv Preprint, arXiv: 1611.01603.
[14] Gong Y, Bowman S R.Ruminating Reader: Reasoning with Gated Multi-Hop Attention[OL]. arXiv Preprint, arXiv: 1704.07415.
[15] Zhang J, Zhu X, Chen Q, et al.Exploring Question Understanding and Adaptation in Neural-network Based Question Answering[OL]. arXiv Preprint, arXiv: 1703.04617.
[16] Shen Y, Huang P, Gao J, et al.ReasoNet: Learning to Stop Reading in Machine Comprehension[C]// Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2017: 1047-1055.
[17] Hu M, Peng Y, Huang Z, et al.Mnemonic Reader for Machine Comprehension[OL]. arXiv Preprint, arXiv: 1705.02798.
[18] Cui Y, Liu T, Chen Z, et al.Dataset for First Evaluation on Chinese Machine Reading Comprehension[OL]. arXiv Preprint, arXiv: 1709.08299.
[19] 顾明远. 教育大辞典[M]. 上海: 上海教育出版社, 1998.
[19] (Gu Mingyuan.Dictionary of Education[M]. Shanghai: Shanghai Educational Publishing House, 1998.)
[20] 张杰, 魏维. 基于视觉注意力模型的显著性提取[J]. 计算机技术与发展, 2010, 20(11): 109-113.
doi: 10.3969/j.issn.1673-629X.2010.11.027
[20] (Zhang Jie, Wei Wei.Saliency Extraction Based on Visual Attention Model[J]. Computer Technology and Development, 2010, 20(11): 109-113.)
doi: 10.3969/j.issn.1673-629X.2010.11.027
[21] 张家俊, 宗成庆. 神经网络语言模型在统计机器翻译中的应用[J]. 情报工程, 2017, 3(3): 21-28.
doi: 10.3772/j.issn.2095-915x.2017.03.004
[21] (Zhang Jiajun, Zong Chengqing.Application of Neural Network Language Model in Statistical Machine Translation[J]. Technology Intelligence Engineering, 2017, 3(3): 21-28.)
doi: 10.3772/j.issn.2095-915x.2017.03.004
[1] Xia Tian. Extracting Key-phrases from Chinese Scholarly Papers[J]. 数据分析与知识发现, 2020, 4(7): 76-86.
[2] Lv Huakui,Hong Liang,Ma Feicheng. Constructing Knowledge Graph for Financial Equities[J]. 数据分析与知识发现, 2020, 4(5): 27-37.
[3] Xinrui Wang,Yue He. Predicting Stock Market Fluctuations with Social Media Behaviors: Case Study of Sina Finance Blog[J]. 数据分析与知识发现, 2019, 3(11): 108-119.
[4] Yong Cheng,Dekuan Xu,Xueqiang Lv. Automatically Grading Text Difficulty with Multiple Features[J]. 数据分析与知识发现, 2019, 3(7): 103-112.
[5] Mu Dongmei,Wang Ping,Zhao Danning. Reducing Data Dimension of Electronic Medical Records: An Empirical Study[J]. 数据分析与知识发现, 2018, 2(1): 88-98.
[6] He Yue,Yin Xiaojia,Zhu Chao. Analyzing Characteristics of Weibo Users Based on Their Sentiments and Influences —— Case Study of Cell Phone Brand[J]. 数据分析与知识发现, 2017, 1(10): 12-20.
[7] Hu Zhongyi,Wang Chaoqun,Wu Jiang. Identifying Phishing Websites with Multiple Online Data Sources[J]. 数据分析与知识发现, 2017, 1(6): 47-55.
[8] Zhang Xiaoojuan. Analyzing Dynamic Informational, Navigational and Transactional Online Queries[J]. 数据分析与知识发现, 2017, 1(4): 9-19.
[9] Zeng Jin,Lu Wei,Ding Heng,Chen Haihua. Modeling User’s Interests Based on Image Semantics[J]. 数据分析与知识发现, 2017, 1(4): 76-83.
[10] Wang Ting,Gao Ying,Liu Jingwei. Linking Chinese Open Data at Schema-Level[J]. 数据分析与知识发现, 2017, 1(2): 47-57.
[11] Xia Tian. Extracting Keywords with Modified TextRank Model[J]. 数据分析与知识发现, 2017, 1(2): 28-34.
[12] Sun Yi'nan, Ku Liping, Song Xiufang, Liu Jingjing, Jiang Xian. The Policy Research and Analysis of Subject Data Repository ——Cases Study of Life Sciences[J]. 现代图书情报技术, 2015, 31(12): 13-20.
[13] Ren Ni, Zhou Jiannong. The Discovery and Evaluation of Research Team Under the Mode of Weighted Co-Author Network[J]. 现代图书情报技术, 2015, 31(9): 68-75.
[14] Fu Honghu, Zhang Zhixiong, Liu Jianhua, Qian Li, Wang Ying. Construction of STKOS Term Publishing and Sharing Service Platform[J]. 现代图书情报技术, 2015, 31(9): 76-81.
[15] Wu Ni, Zhao Pengwei, Qin Chunxiu. Microblog Hotspot Detection Based on Semantic Analysis and Similarity Strength[J]. 现代图书情报技术, 2015, 31(5): 57-64.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn