|
|
Automatic Scoring for Subjective Questions in Maritime Competency Assessment |
Han Hui(),Liu Xiuwen |
Navigation College, Dalian Maritime University, Dalian 116000, China |
|
|
Abstract [Objective] This paper builds an automatic scoring system for subjective questions in the maritime competency assessment, aiming to reduce the heavy workload and human factors of subjective question scoring. [Methods] Firstly, we used the weighted TextRank algorithm of dependency syntax analysis to extract keywords. Then, we integrated sentence vectors, core words, syntactic components, and dependent structures to judge the similarity between student answers and the standard ones. Third, we constructed a set of special negative words for maritime affairs to judge the semantic opposition between the student’s answer and the standard answer. Finally, we gave each answer an objective score. [Results] We examined our method with multiple sets of different subjective questions, and found the average score difference between the automatic score and the manual scoring was 0.21, with a deviation rate of 4.20%. [Limitations] More research is needed to improve the processing of long and complex sentences. [Conclusions] The proposed algorithm could effectively evaluate subjective questions in the maritime competency assessment.
|
Received: 30 November 2020
Published: 15 September 2021
|
|
Fund:Ministry of Industry and Information Technology([2018]No.473);Fundamental Research Funds for the Central Universities(3132019312) |
Corresponding Authors:
Han Hui ORCID:0000-0002-3053-0614
E-mail: hanh1201@126.com
|
[1] |
中华人民共和国海事局. STCW公约马尼拉修正案履约指南[M]. 大连: 大连海事大学出版社, 2010.
|
[1] |
( China Maritime Safety Administration. Guidelines for the Implementation of the Manila Amendment to the STCW Convention[M]. Dalian: Dalian Maritime University Press, 2010.)
|
[2] |
Page E B. Project Essay Grade: PEG[A]//Shermis M D, Burstein J. Automated Essay Scoring: A Cross-disciplinary Perspective[M]. Lawrence Erlbaum Associates Publishers, 2003: 43-54.
|
[3] |
Foltz P W, Streeter L E, Lochbaum K E, et al. Implementation and Applications of the Intelligent Essay Assessor[A]//Shermis M D, Burstein J. Handbook of Automated Essay Evaluation[M]2013:68-88.
|
[4] |
Burstein J, Tetreault J, Madnani N. The E-rater Automated Essay Scoring System[A]//Handbook of Automated Essay Evaluation: Current Applications and New Directions[M]. New York: Routledge, 2013: 55-67.
|
[5] |
Burrows S, Gurevych I, Stein B. The Eras and Trends of Automatic Short Answer Grading[J]. International Journal of Artificial Intelligence in Education, 2015, 25:60-117.
|
[6] |
Chapelle C A, Chung Y R. The Promise of NLP and Speech Processing Technologies in Language Assessment[J]. Language Testing, 2010, 27(3):301-315.
doi: 10.1177/0265532210364405
|
[7] |
Vij S, Tayal D, Jain A. A Machine Learning Approach for Automated Evaluation of Short Answers Using Text Similarity Based on WordNet Graphs[J]. Wireless Personal Communications, 2020, 111(2):1271-1282.
doi: 10.1007/s11277-019-06913-x
|
[8] |
高思丹. 基于自然语言理解的主观试题自动批改技术的研究与初步实现[D]. 南京: 南京大学, 2003.
|
[8] |
( Gao Sidan. Research and Preliminary Realization of Subjective Examination Questions Automatic Correction Technology Based on Natural Language Understanding[D]. Nanjing: Nanjing University, 2003.)
|
[9] |
孟爱国, 卜胜贤, 李鹰, 等. 一种网络考试系统中主观题自动评分的算法设计与实现[J]. 计算机与数字工程, 2005, 33(7):147-150.
|
[9] |
( Meng Aiguo, Bu Shengxian, Li Ying, et al. Design and Implementation of an Automated Assessment Algorithm for Subjective Tests in Network Examination System[J]. Computer and Digital Engineering, 2005, 33(7):147-150.)
|
[10] |
李辉阳, 韩忠愿. 有限领域简述文字的自动判读及其在CAI中的应用[J]. 计算机工程与应用, 2002, 38(8):76-78.
|
[10] |
( Li Huiyang, Han Zhongyuan. Auto-Judging to the Simple Description in the Specific Field and Its Application in CAI[J]. Computer Engineering and Applications, 2002, 38(8):76-78.)
|
[11] |
李学俊. 基于人工智能的主观题自动评分算法实现[J]. 江南大学学报(自然科学版), 2009, 8(3):292-295.
|
[11] |
( Li Xuejun. Algorithm Realization of the Subjective Question’s Auto-Marking Based on the Artificial Intelligence[J]. Journal of Jiangnan University (Natural Science Edition), 2009, 8(3):292-295.)
|
[12] |
张胜楠. 基于文本相似度的短文本主观题自动评分方法研究[D]. 武汉: 武汉理工大学, 2017.
|
[12] |
( Zhang Shengnan. Research on Automatic Scoring Method of Short Text Subjective Questions Based on Text Similarity[D]. Wuhan: Wuhan University of Technology, 2017.)
|
[13] |
代钰琴. 智能评卷技术的应用研究[D]. 绵阳: 西南科技大学, 2019.
|
[13] |
( Dai Yuqin. Research on the Application of Intelligent Marking Technology[D]. Mianyang: Southwest University of Science and Technology, 2019.)
|
[14] |
Zhang W N, Ming Z Y, Zhang Y, et al. The Use of Dependency Relation Graph to Enhance the Term Weighting in Question Retrieval[C]// Proceedings of COLING 2012: Technical Papers. 2012: 3105-3120.
|
[15] |
Mihalcea R, Tarau P. TextRank: Bringing Order into Texts[C]// Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing. 2004: 404-411.
|
[16] |
夏天. 词向量聚类加权TextRank的关键词抽取[J]. 数据分析与知识发现, 2017, 1(2):28-34.
|
[16] |
( Xia Tian. Extracting Keywords with Modified TextRank Model[J]. Data Analysis and Knowledge Discovery, 2017, 1(2):28-34.)
|
[17] |
Mikolov T, Sutskever L, Chen K, et al. Distributed Representations of Words and Phrases and Their Compositionality[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems. 2013: 3111-3119.
|
[18] |
Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[C]// Proceeding of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2019: 4171-4186.
|
[19] |
麦范金, 王挺. 语义对立度及其计算模型的研究[J]. 中文信息学报, 2008, 22(4):39-42.
|
[19] |
( Mai Fanjin, Wang Ting. A Computational Model of the Contrary Degree Between Two Words[J]. Journal of Chinese Information Processing, 2008, 22(4):39-42.)
|
[20] |
梁晓诚, 岳晓光, 麦范金, 等. 基于自然语言处理的主观题评分算法研究[J]. 昆明理工大学学报(理工版), 2010, 35(2):81-84.
|
[20] |
( Liang Xiaocheng, Yue Xiaoguang, Mai Fanjin, et al. Algorithm Research of Subjective Question Assessment Based on Natural Language Processing[J]. Journal of Kunming University of Science and Technology (Science and Technology Edition), 2010, 35(2):81-84.)
|
[21] |
Peng B, Wang J, Zhang X J. Adversarial Learning of Sentiment Word Representations for Sentiment Analysis[J]. Information Sciences, 2020, 541:426-441.
doi: 10.1016/j.ins.2020.06.044
|
[22] |
Al-Khalifa H S, Al-Yahya M M, Bahanshal A, et al. SemQ: A Proposed Framework for Representing Semantic Opposition in the Holy Quran Using Semantic Web Technologies[C]// Proceedings of 2009 International Conference on the Current Trends in Information Technology. DOI: 10.1109/CTIT.2009.5423145.
doi: 10.1109/CTIT.2009.5423145
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|