1School of Information, North China University of Technology, Beijing 100144, China 2CNONIX National Standard Application and Promotion Laboratory, Beijing 100144, China
[Objective] This paper proposes a question generation model based on sememe knowledge and bidirectional attention flow, aiming to improve the semantics of the questions. [Methods] We developed two strategies to enhance semantics: (I) By integrating the external knowledge of sememe in the embedding layer, we captured the semantic knowledge with a smaller granularity than word vectors, and then enhanced the semantic features of the text itself. In addition, we obtained an expanded sememe knowledge base that is more in line with the semantics of the contextual text through the cosine similarity algorithm. It helped us filter out the sememes creating semantic noise in the original knowledge base, and recommended semantically compliant sememe sets for words labeled with non-semantic origins. (II) We enhanced the semantic representation between texts and answers by incorporating a bidirectional attention flow after the encoding layer. [Results] We evaluated our model with the SQuAD1.1 dataset, and the Bleu_1, Bleu_2, Bleu_3, and Blue_4 reached 46.70%, 31.07%, 22.90%, and 17.48%, respectively. The proposed model outperformed the baseline models. [Limitations] With the bidirectional attention flow, the model needs to extract features of paragraph texts and questions, which demands double memory and time to train the model. [Conclusions] Sememe knowledge and bidirectional attention flow could help the proposed model generate higher-quality questions more in line with human language habits.
( Wu Yunfang, Zhang Yangsen. A Survey of Question Generation[J]. Journal of Chinese Information Processing, 2021, 35(7): 1-9.)
[2]
Sutskever I, Vinyals O, Le Q V. Sequence to Sequence Learning with Neural Networks[OL]. arXiv Preprint, arXiv: 1409.3215.
[3]
Heilman M, Smith N A. Good Question! Statistical Ranking for Question Generation[C]// Proceedings of the 2010 North American Chapter of the Association for Computational Linguistics Human Language Technologies Conference. 2010: 1-9.
[4]
Mannem P, Prasad R, Joshi A. Question Generation from Paragraphs at UPenn: QGSTEC System Description[C]// Proceedings of the 3rd Workshop on Question Generation. 2010: 84-91.
[5]
Du X Y, Shao J R, Cardie C. Learning to Ask: Neural Question Generation for Reading Comprehension[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA: Association for Computational Linguistics, 2017: 1342-1352.
[6]
Zhou Q, Yang N, Wei F, et al. Neural Question Generation from Text: A Preliminary Study[C]// Proceedings of the 2017 CCF International Conference on Natural Language Processing and Chinese Computing. 2017: 662-671.
[7]
Ma X Y, Zhu Q L, Zhou Y L, et al. Improving Question Generation with Sentence-Level Semantic Matching and Answer Position Inferring[J]. Proceedings of the AAAI Conference on A.pngicial Intelligence, 2020, 34(5): 8464-8471.
( Tan Hongye, Sun Xiuqin, Yan Zhen. Question Generation Model Based on the Answer and Its Contexts[J]. Journal of Chinese Information Processing, 2020, 34(5): 74-81.)
[9]
Pennington J, Socher R, Manning C. GloVe: Global Vectors for Word Representation[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA, USA: Association for Computational Linguistics, 2014: 1532-1543.
[10]
Niu Y L, Xie R B, Liu Z Y, et al. Improved Word Representation Learning with Sememes[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA: Association for Computational Linguistics, 2017: 2049-2058.
( Yan Qiang, Zhang Xiaoyan, Zhou Simin. Extracting Keywords Based on Sememe Similarity[J]. Data Analysis and Knowledge Discovery, 2021, 5(4): 80-89.)
[12]
Seo M, Kembhavi A, Farhadi A, et al. Bidirectional Attention Flow for Machine Comprehension[OL]. arXiv Preprint, arXiv: 1611.01603.
[13]
Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need[OL]. arXiv Preprint, arXiv: 1706.03762.
[14]
Luong T, Pham H, Manning C D. Effective Approaches to Attention-Based Neural Machine Translation[C]// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA, USA: Association for Computational Linguistics, 2015: 1412-1421.
[15]
Zhao Y, Ni X C, Ding Y Y, et al. Paragraph-Level Neural Question Generation with Maxout Pointer and Gated Self-Attention Networks[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA, USA: Association for Computational Linguistics, 2018: 3901-3910.
[16]
Gu J T, Lu Z D, Li H, et al. Incorporating Copying Mechanism in Sequence-to-Sequence Learning[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA: Association for Computational Linguistics, 2016: 1631-1640.
[17]
See A, Liu P J, Manning C D. Get to the Point: Summarization with Pointer-Generator Networks[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA: Association for Computational Linguistics, 2017: 1073-1087.
[18]
Gulcehre C, Ahn S, Nallapati R, et al. Pointing the Unknown Words[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA: Association for Computational Linguistics, 2016: 140-149.
[19]
Kim Y, Lee H, Shin J, et al. Improving Neural Question Generation Using Answer Separation[OL]. arXiv Preprint, arXiv: 1809.02393.
[20]
Wang W H, Yang N, Wei F R, et al. Gated Self-Matching Networks for Reading Comprehension and Question Answering[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA: Association for Computational Linguistics, 2017: 189-198.
