Classification Model for Scholarly Articles Based on Improved Graph Neural Network
Huang Xuejian1,2,Liu Yuyang3,Ma Tinghuai1()
1College of Computer and Software, Nanjing University of Information Science & Technology, Nanjing 210044, China 2VR College of Modern Industry, Jiangxi University of Finance and Economics, Nanchang 330013, China 3College of Humanities, Jiangxi University of Finance and Economics, Nanchang 330013, China
[Objective] This paper tries to address the over-smoothing issues of the traditional graph neural network, and then realizes the weight adaptive allocation of different depths and neighbors, aiming to improve the performance of academic literature classification. [Methods] We proposed an improved graph neural network model for academic paper classification. First, with the help of multi-head attention mechanism, the new model learned a variety of related features among documents, and adaptively distributing the weights of different neighbor nodes. Then, based on the residual network structure, the model aggregated outputs of each layer node, and provided the learning of adaptive aggregation radius. Finally, with the help of improved graph neural network, the model learned feature representation of each node in the paper citation graph, which was input into the multi-layer fully connected network to obtain the final classification. [Results] We examined our model on large-scale real datasets. The accuracy of our model reached 0.61, which is 0.04 and 0.14 higher than those of the GCN and Transformer models. [Limitations] More research is needed to improve the classification accuracy of small categories and difficult to distinguish samples. [Conclusions] The improved graph neural network can effectively conduct classification for academic articles.
黄学坚, 刘雨飏, 马廷淮. 基于改进型图神经网络的学术论文分类模型*[J]. 数据分析与知识发现, 2022, 6(10): 93-102.
Huang Xuejian, Liu Yuyang, Ma Tinghuai. Classification Model for Scholarly Articles Based on Improved Graph Neural Network. Data Analysis and Knowledge Discovery, 2022, 6(10): 93-102.
Shu F, Julien C A, Zhang L, et al. Comparing Journal and Paper Level Classifications of Science[J]. Journal of Informetrics, 2019, 13(1): 202-225.
doi: 10.1016/j.joi.2018.12.005
(Zhang Chengzhi, Li Zhuo, Chu Heting. Using Full Content to Automatically Classify the Research Methods of Academic Articles[J]. Journal of the China Society for Scientific and Technical Information, 2020, 39(8): 852-862.)
[3]
Sethares W A, Ingle A, Krč T, et al. Eigentextures: An SVD Approach to Automated Paper Classification[C]// Proceedings of the 48th Asilomar Conference on Signals, Systems and Computers. IEEE, 2014: 1109-1113.
(Wu Yongliang, Zhao Shuliang, Li Changjing, et al. Text Classification Method Based on TF-IDF and Cosine Similarity[J]. Journal of Chinese Information Processing, 2017, 31(5): 138-145.)
(Liao Liefa, Le Fugang, Zhu Yalan. The Application of LDA Model in Patent Text Classification[J]. Journal of Modern Information, 2017, 37(3): 35-39.)
doi: 10.3969/j.issn.1008-0821.2017.03.007
[6]
Kim S W, Gil J M. Research Paper Classification Systems Based on TF-IDF and LDA Schemes[J]. Human-Centric Computing and Information Sciences, 2019, 9: 30.
doi: 10.1186/s13673-019-0192-7
(Liu Liu, Wang Dongbo. Identifying Interdisciplinary Social Science Research Based on Article Classification[J]. Data Analysis and Knowledge Discovery, 2018, 2(3): 30-38.)
(Dong Fang, Liu Yufei, Zhou Yuan. Prediction of Emerging Technologies Based on LDA-SVM Multi-Class Abstract of Paper Classification[J]. Journal of Intelligence, 2017, 36(7): 40-45.)
(Wang Hao, Ye Peng, Deng Sanhong. The Application of Machine-Learning in the Research on Automatic Categorization of Chinese Periodical Articles[J]. New Technology of Library and Information Service, 2014(3): 80-87.)
(Xue Feng, Hu Yue, Xia Shuai, et al. Research on Short Text Classification Based on Paper Title and Abstract[J]. Journal of Hefei University of Technology(Natural Science), 2018, 41(10): 1343-1349.)
[11]
Koutsomitropoulos D A, Andriopoulos A D. Thesaurus-Based Word Embeddings for Automated Biomedical Literature Classification[J]. Neural Computing & Applications, 2022, 34(2): 937-950.
(Lyu Lucheng, Han Tao, Zhou Jian, et al. Research on the Method of Chinese Patent Automatic Classification Based on Deep Learning[J]. Library and Information Service, 2020, 64(10): 75-85.)
doi: 10.13266/j.issn.0252-3116.2020.10.009
(Deng Sanhong, Fu Yuyangzi, Wang Hao. Multi-Label Classification of Chinese Books with LSTM Model[J]. Data Analysis and Knowledge Discovery, 2017, 1(7): 52-60.)
(Xu Tongyang, Yin Kai. Text Classification of Digital Library Based on Deep Learning[J]. Information Science, 2019, 37(10): 13-19.)
[15]
Xu H T, Dong M, Zhu D X, et al. Text Classification with Topic-Based Word Embedding and Convolutional Neural Networks[C]// Proceedings of the 7th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics. 2016: 88-97.
(Wang Xinyun, Wang Hao, Deng Sanhong, et al. Classification of Academic Papers for Periodical Selection[J]. Data Analysis and Knowledge Discovery, 2020, 4(7): 96-109.)
(Xie Hongling, Feng Guohe, He Weilin. Research on Semantic Classification of Scientific and Technical Literature Based on Deep Learning[J]. Information Studies: Theory & Application, 2018, 41(11): 149-154.)
[18]
Peters M E, Neumann M, Iyyer M, et al. Deep Contextualized Word Representations[OL]. arXiv Preprint, arXiv: 1802.05365.
[19]
Radford A, Narasimhan K, Salimans T, et al. Improving Language Understanding by Generative Pre-training[EB/OL].[2021-11-30]. https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf.
[20]
Devlin J, Chang M W, Lee K, et al. BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding[OL]. arXiv Preprint, arXiv: 1810.04805.
(Ni Bin, Lu Xiaolei, Tong Yiqi, et al. Automated Journal Text Classification Based on Capsule Neural Network[J]. Journal of Nanjing University(Natural Science), 2021, 57(5): 750-756.)
(Liu Lei, Xu Jie, Zhou Yong. The Study on ERBERT-GRU Chinese Book Classification Method Based on Knowledge Enhancement[J]. Journal of Jiangxi Normal University(Natural Science Edition), 2021, 45(3): 299-304.)
[24]
Tezgider M, Yildiz B, Aydin G. Text Classification Using Improved Bidirectional Transformer[J]. Concurrency and Computation: Practice and Experience, 2022, 34(9): e6486.
[25]
Gori M, Monfardini G, Scarselli F. A New Model for Learning in Graph Domains[C]// Proceedings of the IEEE International Joint Conference on Neural Networks. IEEE, 2005: 729-734.
[26]
Scarselli F, Gori M, Tsoi A C, et al. The Graph Neural Network Model[J]. IEEE Transactions on Neural Networks, 2009, 20(1): 61-80.
doi: 10.1109/TNN.2008.2005605
pmid: 19068426
[27]
Bruna J, Zaremba W, Szlam A, et al. Spectral Networks and Locally Connected Networks on Graphs[OL]. arXiv Preprint, arXiv: 1312.6203.
[28]
Kipf T N, Welling M. Semi-Supervised Classification with Graph Convolutional Networks[OL]. arXiv Preprint, arXiv: 1609.02907.
[29]
Hamilton W L, Ying R, Leskovec J. Inductive Representation Learning on Large Graphs[OL]. arXiv Preprint, arXiv: 1706.02216.
[30]
Velikovi P, Cucurull G, Casanova A, et al. Graph Attention Networks[OL]. arXiv Preprint, arXiv: 1710.10903.
[31]
He K M, Zhang X Y, Ren S Q, et al. Deep Residual Learning for Image Recognition[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2016: 770-778.
[32]
Zhang Z L, Sabuncu M R. Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels[OL]. arXiv Preprint, arXiv: 1805.07836.
