1School of Management Science and Information Engineering, Jilin University of Finance and Economics, Changchun 130117, China 2Business Big Data Research Center of Jilin Province, Changchun 130117, China
[Objective] This paper constructs an expression data set of engagement degrees and designs a joint evaluation model for students’ class engagement. It addresses the issues of lacking relevant expression data sets and the low accuracy of the existing models. [Methods] We collected data based on actual online classes and constructed an expression dataset suitable for engagement recognition. Then, we designed an improved VGG model to evaluate the dataset and recognize student engagement. Third, we combined the expression and face scores to establish a joint evaluation model for students’ engagement and calculated the tested students’ actual class engagement scores. [Results] We adjusted and verified the network structure through parameter tuning optimization for engagement expression recognition. The improved model VGG16+Dense+Dropout(lr=1e-5) had the highest accuracy among the four compared model architectures, reaching over 92%. The joint engagement score is more accurate for engagement evaluation than the single expression engagement score. [Limitations] We did not include more ablation studies in training the model; more research is needed to explore the deeper neural networks. [Conclusions] The dataset of W-AttLe is suitable for evaluating students’ class engagement. The proposed joint engagement evaluation model outperforms the single index model. The proposed weighted test scheme combining knowledge point test and self-test of comprehension degree validates the joint engagement degree model.
王楠, 王淇. 基于深度学习的学生课堂专注度测评方法*[J]. 数据分析与知识发现, 2023, 7(6): 123-133.
Wang Nan, Wang Qi. Evaluating Student Engagement with Deep Learning. Data Analysis and Knowledge Discovery, 2023, 7(6): 123-133.
Howard W R. Pattern Recognition and Machine Learning[J]. Kybernetes, 2007, 36(2): 275.
doi: 10.1108/03684920710743466
[2]
LeCun Y, Bengio Y, Hinton G. Deep Learning[J]. Nature, 2015, 521(7553): 436-444.
doi: 10.1038/nature14539
[3]
Coan J A, Allen J J B. Frontal EEG Asymmetry as a Moderator and Mediator of Emotion[J]. Biological Psychology, 2004, 67(1-2):7-50.
doi: 10.1016/j.biopsycho.2004.03.002
pmid: 15130524
[4]
Day M. Exploiting Facial Landmarks for Emotion Recognition in the Wild[OL]. arXiv Preprint, arXiv: 1603.09129.
[5]
LeCun Y, Bottou L, Bengio Y, et al. Gradient-Based Learning Applied to Document Recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
doi: 10.1109/5.726791
[6]
Dinesh D, S. A N, Bijlani K. Student Analytics for Productive Teaching/Learning[C]// Proceedings of International Conference on Information Science. 2017: 97-102.
[7]
Butko N J, Theocharous G, Philipose M, et al. Automated Facial Affect Analysis for One-on-One Tutoring Applications[C]// Proceedings of 2011 IEEE International Conference on Automatic Face & Gesture Recognition. 2011: 382-387.
[8]
Hedges J H, Adolph K E, Amso D, et al. Play, Attention, and Learning: How Do Play and Timing Shape the Development of Attention and Influence Classroom Learning?[J]. Annals of the New York Academy of Sciences, 2013, 1292(1): 1-20.
doi: 10.1111/nyas.2013.1292.issue-1
[9]
Smith L B, Colunga E, Yoshida H. Knowledge as Process: Contextually Cued Attention and Early Word Learning[J]. Cognitive Science, 2010, 34(7): 1287-1314.
doi: 10.1111/j.1551-6709.2010.01130.x
[10]
Steinmayr R, Ziegler M, Träuble B. Do Intelligence and Sustained Attention Interact in Predicting Academic Achievement?[J]. Learning and Individual Differences, 2010, 20(1): 14-18.
doi: 10.1016/j.lindif.2009.10.009
[11]
Chen C M, Huang S H. Web-Based Reading Annotation System with an Attention-Based Self-Regulated Learning Mechanism for Promoting Reading Performance[J]. British Journal of Educational Technology, 2014, 45(5): 959-980.
doi: 10.1111/bjet.2014.45.issue-5
[12]
伍万杰. 基于姿态分析的学生学习状态监测方法研究与实现[D]. 重庆: 西南大学, 2021.
[12]
(Wu Wanjie. Research and Implementation of Students’ Learning State Monitoring Method Based on Posture Analysis[D]. Chongqing: Southwest University, 2021.)
[13]
Zaletelj J, Košir A. Predicting Students’ Attention in the Classroom from Kinect Facial and Body Features[J]. EURASIP Journal on Image and Video Processing, 2017, 2017: 80.
doi: 10.1186/s13640-017-0228-8
[14]
段巨力. 基于机器视觉的学生上课专注度的分析评测系统[D]. 杭州: 浙江工商大学, 2018.
[14]
(Duan Juli. Evaluation and Evaluation System of Students’ Attentiveness Based on Machine Vision[D]. Hangzhou: Zhejiang Gongshang University, 2018.)
(Zuo Guocai, Wang Haidong, Chen Linshu, et al. Evaluation of Modern Apprenticeship Learning Effect Based on Face Recognition Technology[J]. Intelligent Computer and Applications, 2019, 9(2): 116-118.)
(Zhong Machi, Zhang Junlang, Lan Yangbo, et al. Study on Online Education Focus Degree Based on Face Detection and Fuzzy Comprehensive Evaluation[J]. Computer Science, 2020, 47(S2): 196-203.)
(He Xiuling, Gao Qian, Li Yangyang, et al. Spontaneous Learning Facial Expression Recognition Based on Deep Learning[J]. Computer Applications and Software, 2019, 36(3): 180-186.)
[18]
Soloviev V. Machine Learning Approach for Student Engagement Automatic Recognition from Facial Expressions[J]. Scientific Publications of the State University of Novi Pazar Series A: Applied Mathematics Informatics and Mechanics, 2018, 10(2): 79-86.
doi: 10.5937/SPSUNP1802079S
[19]
Mehrabian A. Communication Without Words[A]//Communication Theory[M]. London: Routledge, 2017: 193-200.
[20]
Cowen A S, Keltner D. Self-Report Captures 27 Distinct Categories of Emotion Bridged by Continuous Gradients[J]. Proceedings of the National Academy of Sciences of the United States of America, 2017, 114(38): E7900-E7909.
[21]
郭晓旭. 基于微表情识别的学生课堂专注度分析系统研究[D]. 昆明: 云南师范大学, 2019.
[21]
(Guo Xiaoxu. Research on the Analysis System of Students’ Classroom Attention Analysis Based on Micro-expression Recognition[D]. Kunming: Yunnan Normal University, 2019.)
