A Comparative Study of Word Representation Models based on Deep Learning
Yu Chuanming,Wang Manyi,Lin Hongjun,Zhu Xingyu,Huang Tingting,An Lu
(School of Information and Safety Engineering, Zhongnan University of Economics and Law, Wuhan 430073, China)
(School of Statistics and Mathematics, Zhongnan University of Economics and Law, Wuhan 430073, China)
(School of Information Management, Wuhan University, Wuhan 430072, China)
[Objective] This study aims to systematically reveal the principles of the traditional deep representation models and the latest pre-training models and explore their differences in text mining tasks. [Methods] Using the comparative study method, we investigate the differences between the traditional models and the latest models on the six datasets of CR, MR, MPQA, Subj, SST-2 and TREC from the model side and the experimental side. [Results] On the six tasks, the XLNet model achieved the best average F1 value (0.9186) overall, which is better than ELMo (0.8090), BERT(0.8983), Word2Vec(0.7692), GloVe(0.7576) and FastText(0.7506). [Limitations] Due to space limitations, the empirical research focuses on classification tasks of text mining. It has not compared the effects of multiple vocabulary representation methods in machine translation, Q&A and other tasks. [Conclusions] The traditional deep representation learning model and the latest pre-training model still have large effect differences in text mining tasks.
余传明, 王曼怡, 林虹君, 朱星宇, 黄婷婷, 安璐. 基于深度学习的词汇表示模型对比研究
[J]. 数据分析与知识发现, 0, (): 1-.
Yu Chuanming, Wang Manyi, Lin Hongjun, Zhu Xingyu, Huang Tingting, An Lu. A Comparative Study of Word Representation Models based on Deep Learning
. Data Analysis and Knowledge Discovery, 0, (): 1-.