A Prediction Model with Network Representation Learning and Topic Model for Author Collaboration
Zhang Xin1,Wen Yi1,2(),Xu Haiyun1,2
1Chengdu Library and Information Center, Chinese Academy of Sciences, Chengdu 610041, China 2Department of Library, Information and Archives Management, School of Economics and Management, University of Chinese Academy of Sciences, Beijing 100190, China
[Objective] This paper proposes a method to predict scientific collaboration based on the network representation learning and author topic model. [Methods] First, we established the embedding vector representation of authors with the help of network representation learning method. Then, we calculated the structural similarity of authors with cosine similarity. Third, we obtained the topic representation of authors with the author-topic model, and computed the authors’ topic similarity with Hellinger distance. Finally, we linearly merged the two similarity measures, and used the Bayesian optimization method for the hyperparameter selection. [Results] We examined the proposed method with the NIPS datasets and found the best node2vec+ATM model after Bayesian parameter selection. It had an AUC value of 0.9271, which was 0.1856 higher than that of the benchmark model. [Limitations] We did not include the author’s institution and geographic location to the model. [Conclusions] The proposed model utilizes structure and content features to improve the prediction results of network representation learning.
张鑫,文奕,许海云. 一种融合表示学习与主题表征的作者合作预测模型*[J]. 数据分析与知识发现, 2021, 5(3): 88-100.
Zhang Xin,Wen Yi,Xu Haiyun. A Prediction Model with Network Representation Learning and Topic Model for Author Collaboration. Data Analysis and Knowledge Discovery, 2021, 5(3): 88-100.
model human feature neuron spike task architecture response neural brain study input visual region population mechanism activity natural level implement
Topic2 模型推断
model inference process Bayesian structure variable gaussian approach latent distribution probabilistic variational tree datum method likelihood generative posterior Markov graphical
Topic3 数据
datum show result large method well scale propose number set achieve high performance dataset paper algorithm parameter require order experiment
Topic4 模式识别
network miss neural image learn deep object train representation layer convolutional recognition information recurrent code different learning noisy model visual
Topic5 模型计算
algorithm problem function method optimization result gradient convex show bound stochastic loss convergence study guarantee learning online set regret rate
Topic6 分类聚类特征提取
learn feature datum method kernel task approach propose learning label base classification graph problem art clustering dataset cluster metric class
Topic7 采样降维
sample sparse distribution matrix estimate estimation analysis problem estimator point statistical dimensional non provide show high regression low error consider
Topic8 时序挖掘
time state dynamic learn system decision policy optimal search action approach control problem base information reinforcement space user reward value
Table 2 作者-主题模型抽取出来的研究主题
Fig.6 ATM计算得到的作者分布
算法
正确率
召回率
AUC值
原模型
X+ATM
原模型
X+ATM
原模型
连接
X+ATM
ATM
0.003 0
1.000 0
0.500 0
LE
0.966 6
0.940 1
0.449 9
0.594 8
0.709 1
0.628 4
0.768 1
GF
0.996 0
0.993 0
0.008 3
0.046 9
0.505 7
0.608 2
0.521 4
DeepWalk
0.992 2
0.972 3
0.223 8
0.537 4
0.609 2
0.569 3
0.755 5
LINE
0.199 7
0.059 9
0.961 3
0.997 5
0.579 4
0.603 3
0.527 3
node2vec
0.979 5
0.878 8
0.502 1
0.811 4
0.741 5
0.735 0
0.845 2
SDNE
0.543 4
0.432 6
0.561 3
0.726 5
0.552 3
0.552 3
0.579 1
TADW
0.977 6
0.375 1
0.677 3
CANE
0.992 5
0.328 1
0.661 4
Table 3 融合模型的计算结果
迭代次数
α
AUC值
迭代次数
α
AUC值
1
0.417 0
0.862 8
8
0.560 2
0.922 1
2
0.720 3
0.881 7
9
0.592 7
0.922 9
3
0.000 1
0.500 0
10
0.581 3
0.927 1
4
0.302 3
0.777 6
11
0.580 5
0.926 8
5
1.000 0
0.809 7
12
0.580 2
0.926 6
6
0.577 4
0.926 0
13
0.578 8
0.926 3
7
0.599 2
0.924 2
14
0.578 4
0.926 2
Table 4 node2vec+ATM模型贝叶斯优化参数选择过程
Fig.7 贝叶斯优化结果图
模型
最优融合参数
AUC值
LE+ATM
0.427 4
0.830 9
Graph Factorization+ATM
0.219 6
0.676 6
DeepWalk+ATM
0.344 9
0.835 4
LINE+ATM
0.999 9
0.577 5
node2vec+ATM
0.581 3
0.927 1
SDNE+ATM
0.359 2
0.555 5
Table 5 融合模型的最优融合参数值以及相应的AUC值
合作者
Yoshua Bengio
Geoffrey E.Hinton
Yann LeCun
1
Pascal Vincent
Sam T. Roweis
John S. Denker
2
David S. Touretzky
Christopher K. I. Williams
Rob Fergus
3
Samy Bengio
Richard S. Zemel
Corinna Cortes
4
Yann LeCun
Max Welling
Vladimir Vapnik
5
Ruslan R. Salakhutdinov
Ilya Sutskever
Yoshua Bengio
6
Mitsuo Kawato
Ruslan R. Salakhutdinov
Alex Waibel
7
John S. Denker
Brendan J. Frey
Bartlett W. Mel
8
Ilya Sutskever
Peter Dayan
Andrew Zisserman
9
Geoffrey E. Hinton
Yee W. The
Christof Koch
10
Yoram Singer
Lawrence K. Saul
Tomaso Poggio
Table 6 作者合作预测结果
[1]
Newman M E J. Coauthorship Networks and Patterns of Scientific Collaboration[J]. Proceedings of the National Academy of the United States of America, 2004,101(S1):5200-5205.
[2]
Liben‐Nowell D, Kleinberg J. The Link‐Prediction Problem for Social Networks[J]. Journal of the American Society for Information Science and Technology, 2007,58(7):1019-1031.
[3]
吕琳媛. 复杂网络链路预测[J]. 电子科技大学学报, 2010,39(5):651-661.
[3]
( Lv Linyuan. Link Prediction in Complex Networks[J]. Journal of University of Electronic Science and Technology of China, 2010,39(5):651-661.)
[4]
Guns R, Rousseau R. Recommending Research Collaborations Using Link Prediction and Random Forest Classifiers[J]. Scientometrics, 2014,101(2):1461-1473.
[5]
Yan E, Guns R. Predicting and Recommending Collaborations: An Author-, Institution-, and Country-Level Analysis[J]. Journal of Informetrics, 2014,8(2):295-309.
( Wang Zhibing, Han Wenmin, Sun Zhumei, et al. Research on Scientific Collaboration Prediction Based on the Combination of Network Topology and Node Attributes[J]. Information Studies: Theory & Application, 2019,42(8):116-120, 109.)
( Shan Songyan, Wu Zhenxin. Review on the Author Similarity Algorithm in the Field of Author Name Disambiguation and Research Collaboration Prediction[J]. Journal of Northeast Normal University(Natural Science Edition), 2019,51(2):71-80.)
( Zhang Jinzhu, Yu Wenqian, Liu Jingjie, et al. Predicting Research Collaborations Based on Network Embedding[J]. Journal of the China Society for Scientific and Technical Information, 2018,37(2):132-139.)
( Yu Chuanming, Lin Aochen, Zhong Yunci, et al. Scientific Collaboration Recommendation Based on Network Embedding[J]. Journal of the China Society for Scientific and Technical Information, 2019,38(5):500-511.)
[10]
Balasubramanian M, Schwartz E L. The Isomap Algorithm and Topological Stability[J]. Science, 2002,295(5552):7.
pmid: 11778013
[11]
Roweis S T, Saul L K. Nonlinear Dimensionality Reduction by Locally Linear Embedding[J]. Science, 2000,290(5500):2323-2326.
[12]
Belkin M, Niyogi P. Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering[C]// Proceedings of the 14th International Conference on Neural Information Processing Systems: Natural and Synthetic. 2002: 585-591.
[13]
Chen M, Yang Q, Tang X O. Directed Graph Embedding[C]// Proceedings of the 20th International Joint Conference on Artificial Intelligence. 2007: 2707-2712.
[14]
Ahmed A, Shervashidze N, Narayanamurthy S, et al. Distributed Large-Scale Natural Graph Factorization[C]// Proceedings of the 22nd International Conference on World Wide Web. ACM, 2013: 37-48.
[15]
Perozzi B, Al-Rfou R, Skiena S. DeepWalk: Online Learning of Social Representations[C]// Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2014: 701-710.
[16]
Grover A, Leskovec J. node2vec: Scalable Feature Learning for Networks[C]// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2016: 855-864.
[17]
Cao S S, Lu W, Xu Q K. GraRep: Learning Graph Representations with Global Structural Information[C]// Proceedings of the 24th ACM International Conference on Information and Knowledge Management. ACM, 2015: 891-900.
[18]
Tang J, Qu M, Wang M Z, et al. LINE: Large-Scale Information Network Embedding[C]// Proceedings of the 24th International Conference on World Wide Web. 2015: 1067-1077.
[19]
Wang D X, Cui P, Zhu W W. Structural Deep Network Embedding[C]// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2016: 1225-1234.
[20]
Ou M D, Cui P, Pei J, et al. Asymmetric Transitivity Preserving Graph Embedding[C]// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 2016: 1105-1114.
[21]
Kipf T N, Welling M. Variational Graph Auto-Encoders[OL]. arXiv Preprint, arXiv: 1611.07308,2016.
[22]
Wang H W, Wang J, Wang J L, et al. GraphGAN: Graph Representation Learning with Generative Adversarial Nets[J]. IEEE Transactions on Knowledge and Data Engineering, DOI:10.1109/TKDE.2019.2961882.
doi: 10.1109/TKDE.2012.149
pmid: 24693210
[23]
Yang C, Liu Z Y, Zhao D L, et al. Network Representation Learning with Rich Text Information[C]// Proceeding of the 24th International Conference on Artificial Intelligence. 2015: 2111-2117.
[24]
Sun X F, Guo J, Ding X, et al. A General Framework for Content-Enhanced Network Representation Learning[OL]. arXiv Preprint, arXiv: 1610.02906,2016.
[25]
Tu C C, Liu H, Liu Z Y, et al. CANE: Context-Aware Network Embedding for Relation Modeling[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017: 1722-1731.
[26]
Lerer A, Wu L, Shen J J, et al. PyTorch-BigGraph: A Large-scale Graph Embedding System[C]// Proceedings of the Conference on Systems and Machine Learning. 2019.
[27]
Fey M, Lenssen J E. Fast Graph Representation Learning with PyTorch Geometric[OL]. arXiv Preprint, arXiv Preprint, arXiv: 1903.02428,2019.
[28]
Zhu Z C, Xu S Z, Tang J, et al. GraphVite: A High-Performance CPU-GPU Hybrid System for Node Embedding[C]// Proceedings of the World Wide Web Conference. ACM, 2019: 2494-2504.
[29]
Blei D M, Ng A Y, Jordan M I. Latent Dirichlet Allocation[J]. Journal of Machine Learning Research, 2003,3:993-1022.
[30]
Rosen-Zvi M, Griffiths T, Steyvers M, et al. The Author-Topic Model for Authors and Documents[C]// Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence. AUAI Press, 2004: 487-494.
[31]
Snoek J, Larochelle H, Adams R P. Practical Bayesian Optimization of Machine Learning Algorithms[C]// Proceedings of the 25th International Conference on Neural Information Processing Systems. 2012: 2951-2959.
[32]
LeCun Y, Bengio Y, Hinton G. Deep Learning[J]. Nature, 2015,521(7553):436.
doi: 10.1038/nature14539
pmid: 26017442
[33]
Zhang J, Dong Y X, Wang Y, et al. ProNE: Fast and Scalable Network Representation Learning[C]// Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI'19). 2019: 4278-4284
[34]
Qiu J Z, Dong Y X, Ma H, et al. NetSMF: Large-Scale Network Embedding as Sparse Matrix Factorization[C]// Proceedings of the World Wide Web Conference. ACM, 2019: 1509-1520.
