Please wait a minute...
Data Analysis and Knowledge Discovery  2021, Vol. 5 Issue (4): 134-141    DOI: 10.11925/infotech.2096-3467.2020.0714
Current Issue | Archive | Adv Search |
A Multi-Perspective Co-Matching Model for Machine Reading Comprehension
Duan Jianyong1,2(),Wei Xiaopeng1,Wang Hao1,2
1School of Information, North China University of Technology, Beijing 100144, China
2CNONIX National Standard Application and Promotion Laboratory, North China University of Technology, Beijing 100144, China
Download: PDF (873 KB)   HTML ( 12 ) 
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper proposes a model for multiple-choice reading comprehension, and then explores the impacts of question types and answer length on machine reading comprehension. [Methods] First, we used the multi-perspective matching mechanism to obtain the correlation between the articles, questions and candidate answers. Then, we multiplied the correlation and articles to create the vector representation of questions and candidate answers. Third, we extracted sentence-level and document-level features, which were used to select the correct answers. Fourth, we categorized the data based on the question types and answer length. Finally, we analyzed their impacts on the machine’s choice of correct answers. [Results] The accuracy of our model on the RACE-M, RACE-H and RACE datasets reached 72.5%, 63.1% and 66.1% respectively. [Limitations] The multi-perspective matching mechanism has four matching strategies and multiple angles, which makes the model consume a lot of memory and spend longer processing time at the interactive layer. [Conclusions] The proposed model can effectively match articles with questions and answers. The accuracy of the model is more affected by the type of question, not by the length of answer.
Key words： Machine Reading Comprehension      Multi-Choice      Multi-Perspective Matching      Attention Mechanism     
Received: 21 July 2020      Published: 17 May 2021
ZTFLH:  分类号: TP391  
Fund:National Natural Science Foundation of China(61672040);National Natural Science Foundation of China(61972003)
Corresponding Authors: Duan Jianyong     E-mail: duanjy@ncut.edu.cn
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Jianyong Duan
Xiaopeng Wei
Hao Wang

Cite this article:

Duan Jianyong,Wei Xiaopeng,Wang Hao. A Multi-Perspective Co-Matching Model for Machine Reading Comprehension. Data Analysis and Knowledge Discovery, 2021, 5(4): 134-141.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2020.0714     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2021/V5/I4/134

Overall Structure of the Model
Different Matching Strategies
数据集 RACE-M RACE-H RACE
子集 Train Dev Test Train Dev Test Train Dev Test all
文章数（篇） 6 409 368 362 18 728 1 021 1 045 25 137 1 389 1 407 27 933
问题数（个） 25 421 1 436 1 436 62 445 3 451 3 498 87 866 4 887 4 934 97 687
Information of the Dataset
模型 RACE-M RACE-H RACE
SAR 44.2 43.0 43.2
GA 43.7 44.2 44.1
ElimiNet - - 44.7
HAF 45.3 47.9 47.2
MUSIC 51.5 45.7 47.4
HCM 55.8 48.2 50.4
MRU 57.7 47.4 50.4
BERTbase
BERTbase+MPCM		 71.1
72.5		 62.3
63.1		 65.0
66.1
Comparison of Experimental Results
问题类型 blank who when where what why which how title others
问题数量（个） 44 577 1 149 733 1 064 16 419 3 932 10 697 3 397 1 495 5 801
准确率（%） 67.3 55.4 56.1 64.3 60.9 67.2 60.4 63.5 66.7 60.5
The Number and Accuracy of Different Types of Questions
Accuracy of Questions with Different Length Answers
[1] Tang M, Cai J, Zhuo H. Multi-Matching Network for Multiple Choice Reading Comprehension[C]// Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019,33:7088-7095.
[2] Zhu H, Wei F, Qin B, et al. Hierarchical Attention Flow for Multiple-Choice Reading Comprehension[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018.
[3] Lai G, Xie Q, Liu H, et al. Race: Large-Scale Reading Comprehension Dataset from Examinations[C]// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017: 795-804.
[4] Hermann K M, Kocisky T, Grefenstette E, et al. Teaching Machines to Read and Comprehend[C]// Proceedings of the 28th International Conference on Neural Information Processing Systems. 2015: 1693-1701.
[5] Nguyen T, Rosenberg M, Song X, et al. MS MARCO: A Human-Generated MAchine Reading Comprehension Dataset[OL]. arXiv Preprint, arXiv: 1611. 09268.
[6] Rajpurkar P, Zhang J, Lopyrev K, et al. SQuAD: 100,000+ Questions for Machine Comprehension of Text[C]// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2016: 2383-2392.
[7] Kadlec R, Schmid M, Bajgar O, et al. Text Understanding with the Attention Sum Reader Network[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016: 908-918.
[8] Dhingra B, Liu H, Yang Z, et al. Gated-Attention Readers for Text Comprehension[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017: 1832-1846.
[9] Seo M, Kembhavi A, Farhadi A, et al. Bidirectional Attention Flow for Machine Comprehension[OL]. arXiv Preprint, arXiv: 1611. 01603.
[10] Wang W, Yang N, Wei F, et al. Gated Self-Matching Networks for Reading Comprehension and Question Answering[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017: 189-198.
[11] Chaturvedi A, Pandit O, Garain U. CNN for Text-Based Multiple Choice Question Answering[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018: 272-277.
[12] Wang S, Yu M, Chang S, et al. A Co-Matching Model for Multi-Choice Reading Comprehension[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2018: 746-751.
[13] Wang Z, Hamza W, Florian R. Bilateral Multi-Perspective Matching for Natural Language Sentences[C]// Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017: 4144-4150.
[14] Devlin J, Chang M, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics. 2019: 4171-4186.
[15] Chen D, Bolton J, Manning C D. A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany. 2016: 2358-2367.
[16] Parikh S, Sai A B, Nema P, et al. ElimiNet: A Model for Eliminating Options for Reading Comprehension with Multiple Choice Questions[C]// Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2018: 4272-4278.
[17] Xu Y, Liu J, Gao J, et al. Towards Human-Level Machine Reading Comprehension: Reasoning and Inference with Multiple Strategies[OL]. arXiv Preprint, arXiv: 1711. 04964.
[18] Tay Y, Tuan L A, Hui S C. Multi-range Reasoning for Machine Comprehension[OL]. arXiv Preprint, arXiv: 180.09074.
[1] Yang Hanxun, Zhou Dequn, Ma Jing, Luo Yongcong. Detecting Rumors with Uncertain Loss and Task-level Attention Mechanism[J]. 数据分析与知识发现, 2021, 5(7): 101-110.
[2] Xie Hao,Mao Jin,Li Gang. Sentiment Classification of Image-Text Information with Multi-Layer Semantic Fusion[J]. 数据分析与知识发现, 2021, 5(6): 103-114.
[3] Yin Pengbo,Pan Weimin,Zhang Haijun,Chen Degang. Identifying Clickbait with BERT-BiGA Model[J]. 数据分析与知识发现, 2021, 5(6): 126-134.
[4] Han Pu,Zhang Zhanpeng,Zhang Mingtao,Gu Liang. Normalizing Chinese Disease Names with Multi-feature Fusion[J]. 数据分析与知识发现, 2021, 5(5): 83-94.
[5] Wang Yuzhu,Xie Jun,Chen Bo,Xu Xinying. Multi-modal Sentiment Analysis Based on Cross-modal Context-aware Attention[J]. 数据分析与知识发现, 2021, 5(4): 49-59.
[6] Jiang Cuiqing,Wang Xiangxiang,Wang Zhao. Forecasting Car Sales Based on Consumer Attention[J]. 数据分析与知识发现, 2021, 5(1): 128-139.
[7] Huang Lu,Zhou Enguo,Li Daifeng. Text Representation Learning Model Based on Attention Mechanism with Task-specific Information[J]. 数据分析与知识发现, 2020, 4(9): 111-122.
[8] Yin Haoran,Cao Jinxuan,Cao Luzhe,Wang Guodong. Identifying Emergency Elements Based on BiGRU-AM Model with Extended Semantic Dimension[J]. 数据分析与知识发现, 2020, 4(9): 91-99.
[9] Shi Lei,Wang Yi,Cheng Ying,Wei Ruibin. Review of Attention Mechanism in Natural Language Processing[J]. 数据分析与知识发现, 2020, 4(5): 1-14.
[10] Xue Fuliang,Liu Lifang. Fine-Grained Sentiment Analysis with CRF and ATAE-LSTM[J]. 数据分析与知识发现, 2020, 4(2/3): 207-213.
[11] Qi Ruihua,Jian Yue,Guo Xu,Guan Jinghua,Yang Mingxin. Sentiment Analysis of Cross-Domain Product Reviews Based on Feature Fusion and Attention Mechanism[J]. 数据分析与知识发现, 2020, 4(12): 85-94.
[12] Xu Tongtong,Sun Huazhi,Ma Chunmei,Jiang Lifen,Liu Yichen. Classification Model for Few-shot Texts Based on Bi-directional Long-term Attention Features[J]. 数据分析与知识发现, 2020, 4(10): 113-123.
[13] Yuemin Wu,Ganggui Ding,Bin Hu. Extracting Relationship of Agricultural Financial Texts with Attention Mechanism[J]. 数据分析与知识发现, 2019, 3(5): 86-92.
[14] Yuman Li,Zhibo Chen,Fu Xu. Classifying Texts with KACC Model[J]. 数据分析与知识发现, 2019, 3(10): 89-97.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626，82624938   E-mail:jishu@mail.las.ac.cn