Please wait a minute...
Data Analysis and Knowledge Discovery  2018, Vol. 2 Issue (1): 29-40    DOI: 10.11925/infotech.2096-3467.2017.0715
Orginal Article Current Issue | Archive | Adv Search |
Optimizing LDA Model with Various Topic Numbers: Case Study of Scientific Literature
Wang Tingting1,2(), Han Man1,2, Wang Yu1
1(School of Statistics, Huaqiao University, Xiamen 361021, China)
2(Center for Modern Applied Statistics and Large Data Research, Huaqiao University, Xiamen 361021,China)
Download: PDF (1272 KB)   HTML ( 8
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper proposes a K-wrLDA model based on adaptive clustering, aiming to improve the subject recognition ability of traditional LDA model, and identify the optimal number of selected topics. [Methods] First, we used the LDA and word2vec models to construct the T-WV matrix containing the probability information and the semantic relevance of the subject words. Then, we selected the number of topics based on the evaluation of clustering effects and the pseudo-F statistic. Finally, we compared the topic identification results of the proposed model with the old ones. [Results] The optimal number of topics was 33 for the proposed model, which also has lower level of perplexity than the traditional ones. [Limitations] The sample size needs to be expanded. [Conclusions] The proposed model, which has better recognition rate than the traditional LDA model, could also calculate the optimal number of topics. The new model may be applied to process large corpus in various fields.

Key wordsTopic Model      Word Embedding      Adaptive Clustering      Perplexity     
Received: 20 July 2017      Published: 05 February 2018
ZTFLH:  C816  

Cite this article:

Wang Tingting,Han Man,Wang Yu. Optimizing LDA Model with Various Topic Numbers: Case Study of Scientific Literature. Data Analysis and Knowledge Discovery, 2018, 2(1): 29-40.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2017.0715     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2018/V2/I1/29

K 10 20 50 100 200 300 400 500
$\alpha $ 5 2.5 1 0.5 0.25 0.17 0.13 0.1
相似词 相似度
LDA模型 0.743
主题模型 0.722
概率主题模型 0.707
潜在狄里克雷 0.599
LDA算法 0.556
生成模型 0.512
主题0 主题1 主题2 主题3 主题4 主题5 主题6 主题7 主题8 主题9
情绪 情感 微博 评论 观点 专利 兴趣 词向量 学科 人物简介
新闻推荐 情感分类 推荐 投诉 评论 主题演化 专家 方剂 知识流 电子书
新闻 评论 用户 子话题 情感分析 在线 评分 点击率 克隆代码 子话题
句子 运动 短文本 信息增益 标注 期刊 评论 评分 文献 农业
interest 特征提取 微博用户 产品 观点挖掘 文本流 项目 遥感 分级 电影
读者 评论文本 推荐算法 翻译 软件 中医药 用户 提案 线程 输入
医疗论坛 实体 词汇 正文 合作 文本分割 偏好 主题模型可视化 问句检索 作者
消息传递算法 聚类 个性化推荐 分派 症状 年度 用户兴趣 伪相关反馈 聚类中心 情感
词语 监督 作文 主题分割 借阅 句子 信息检索 帐号 情感摘要 查询推荐
Web服务 句子 协同过滤 情绪 临床 文献 模式 社会化推荐 主题抽取 日志
主题0 主题1 主题2 主题3 主题4 主题5 主题6 主题7 主题8 主题9
评论 专利 问句检索 查询 医疗论坛 随机变量 新闻 视图 教育资源 文本分割
短文本分类 发明人 运动 分布式 舆论 超文本 推荐算法 低质量回帖 视觉单词 任务模型
点击率 投诉 广告投放 word2vec 脑血管病 情感分类 人群 关键词抽取 提案 语义信息
句子 汽车缺陷 实体 矩阵分解 话题检测 文档 用户兴趣 博客 主题模型可视化 数字资源
相似性度量 遥感 关联主题 词聚类 查询 信息熵 用户评论 安全隐患 观点 特征项
词向量 作弊 单机 共享内存 咨询 网络舆情 粒计算 交通 视频 词向量
观点摘要 mixtureLDA 词项 文本建模 语义指纹 主题情感混合模型 online 关键词集 帐号 投放
朴素贝叶斯 词义 投放 消息传递算法 标记 自动应答系统 个性化推荐 隐患 句群 主题特征
引文上下文 用户 相似度算法 线程 文章 标签抽取 新浪微博 查询 语义标注 偏斜
共享主题 兴趣 热点话题 数字 相似矩阵 马尔科夫 调控 句法分析 标注单词 阅读概率
[1] Blei D M, Ng A Y, Jordan M I.Latent Dirichlet Allocation[J]. Journal of Machine Learning Research, 2003, 3: 993-1022.
[2] Blei D M, Lafferty J D.Correlated Topic Models[J]. Advances in Neural Information Processing Systems, 2005, 18: 113-120.
[3] 关鹏, 王曰芬, 傅柱.不同语料下基于 LDA 主题模型的科学文献主题抽取效果分析[J]. 图书情报工作, 2016, 60(2): 112-121.
[3] (Guan Peng, Wang Yuefen, Fu Zhu.Effect Analysis of Scientific Literature Topic Extraction Based on LDA Topic Model with Different Corpus[J].Library and Information Service, 2016, 60(2): 112-121.)
[4] Griffiths T L, Steyvers M.Finding Scientific Topics[J]. Proceedings of the National Academy of Sciences of the United States of America, 2004, 101(S1): 5228-5235.
doi: 10.1073/pnas.0307752101
[5] 石晶, 胡明, 石鑫, 等. 基于LDA模型的文本分割[J]. 计算机学报, 2008, 31(10): 1865-1873.
doi: 10.3321/j.issn:0254-4164.2008.10.022
[5] (Shi Jing, Hu Ming, Shi Xin, et al.Text Segmentation Based on LDA Model[J]. Chinese Journal of Computers, 2008, 31(10): 1865-1873.)
doi: 10.3321/j.issn:0254-4164.2008.10.022
[6] Hajjem M, Latiri C.Combining IR and LDA Topic Modeling for Filtering Microblogs[J]. Procedia Computer Science, 2017, 112: 761-770.
doi: 10.1016/j.procs.2017.08.166
[7] 廖列法, 勒孚刚, 朱亚兰. LDA模型在专利文本分类中的应用[J]. 现代情报, 2017, 37(3): 35-39.
doi: 10.3969/j.issn.1008-0821.2017.03.007
[7] (Liao Liefa, Le Fugang, Zhu Yalan.The Application of LDA Model in Patent Text Classification[J]. Journal of Modern Information, 2017, 37(3): 35-39.)
doi: 10.3969/j.issn.1008-0821.2017.03.007
[8] 刘江华. 一种基于kmeans聚类算法和LDA主题模型的文本检索方法及有效性验证[J]. 情报科学, 2017, 35(2): 16-21.
[8] (Liu Jianghua.A Text Retrieval Method Based on Kmeans Clustering Algorithm and LDA Topic Model and Its Effectiveness[J]. Information Science, 2017, 35(2): 16-21.)
[9] Teh Y, Jordan M, Beal M, et al.Hierarchical Dirichlet Processes[J]. Journal of the American Statistical Association, 2007, 101(476): 1566-1581.
[10] 颜端武, 陶志恒, 李兰彬. 一种基于HDP模型的主题文献自动推荐方法及应用研究[J]. 情报理论与实践, 2016, 39(1): 128-132.
doi: 10.16353/j.cnki.1000-7490.2016.01.023
[10] (Yan Duanwu, Tao Zhiheng, Li Lanbin.A Method of Automatic Recommendation of Subject Documents Based on HDP Model and Its Application[J]. Information Studies: Theory & Application, 2016, 39(1): 128-132.)
doi: 10.16353/j.cnki.1000-7490.2016.01.023
[11] 唐浩浩, 王波, 席耀一, 等. 基于HDP的无监督微博情感倾向性分析[J]. 信息工程大学学报, 2015, 16(4): 463-469.
doi: 10.3969/j.issn.1671-0673.2015.04.014
[11] (Tang Haohao, Wang Bo, Xi Yaoyi, et al.Unsupervised Sentiment Orientation Analysis on Micro-Blogs Based on Hierarchical Dirichlet Processes[J]. Journal of Information Engineering University, 2015, 16(4): 463-469.)
doi: 10.3969/j.issn.1671-0673.2015.04.014
[12] 曹娟, 张勇东, 李锦涛, 等. 一种基于密度的自适应最优LDA模型选择方法[J]. 计算机学报, 2008, 31(10): 1780-1787.
[12] (Cao Juan, Zhang Yongdong, Li Jintao, et al.A Method of Adaptively Selecting Best LDA Model Based on Density[J]. Chinese Journal of Computers, 2008, 31(10): 1780-1787.)
[13] 关鹏, 王曰芬. 科技情报分析中LDA主题模型最优主题数确定方法研究[J]. 现代图书情报技术, 2016(9): 42-49.
[13] (Guan Peng, Wang Yuefen.Research on the Method of Determining the Optimum Topic Number of LDA Topic Model in Scientific and Technical Information Analysis[J]. New Technology of Library and Information Service, 2016(9): 42-49.)
[14] 茆诗松, 王静龙, 濮晓龙. 高等数理统计[M]. 北京: 高等教育出版社, 2006: 446-449.
[14] (Mao Shisong, Wang Jinglong, Pu Xiaolong.Advanced Mathematical Statistics [M]. Beijing: Higher Education Press, 2006: 446-449.)
[15] Hinton G E.Learning Distributed Representations of Concepts[C]//Proceedings of the 8th Annual Conference of the Cognitive Science Society.1986.
[16] Mikolov T, Sutskever I, Chen K, et al.Distributed Representations of Words and Phrases and Their Compositionality[C]//Proceedings of the Neural Information Processing Systems Conference. 2013.
[17] MacQueen J. Some Methods for Classification and Analysis of Multivariate Observations[C]//Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, 1967.
[18] Wei X, Croft W B.LDA-based Document Models for Ad-Hoc Retrieval[C]//Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.ACM, 2006: 178-185.
[19] 王学民. 应用多元分析[M]. 上海: 上海财经大学出版社, 2003: 217-218.
[19] (Wang Xuemin.Applied Multivariate Analysis [M]. Shanghai: Shanghai University of Finance and Economics Press, 2003: 217-218.)
[20] Heinrich G.Parameter Estimation for Text Analysis[R]. vsonix GmbH + University of Leipzig, 2008: 29-30.
[1] Wang Hanxue,Cui Wenjuan,Zhou Yuanchun,Du Yi. Identifying Pathogens of Foodborne Diseases with Machine Learning[J]. 数据分析与知识发现, 2021, 5(9): 54-62.
[2] Huang Mingxuan,Jiang Caoqing,Lu Shoudong. Expanding Queries Based on Word Embedding and Expansion Terms[J]. 数据分析与知识发现, 2021, 5(6): 115-125.
[3] Yi Huifang,Liu Xiwen. Analyzing Patent Technology Topics with IPC Context-Enhanced Context-LDA Model[J]. 数据分析与知识发现, 2021, 5(4): 25-36.
[4] Shen Si,Li Qinyu,Ye Yuan,Sun Hao,Ye Wenhao. Topic Mining and Evolution Analysis of Medical Sci-Tech Reports with TWE Model[J]. 数据分析与知识发现, 2021, 5(3): 35-44.
[5] Zhang Xin,Wen Yi,Xu Haiyun. A Prediction Model with Network Representation Learning and Topic Model for Author Collaboration[J]. 数据分析与知识发现, 2021, 5(3): 88-100.
[6] Zhao Tianzi, Duan Liang, Yue Kun, Qiao Shaojie, Ma Zijuan. Generating News Clues with Biterm Topic Model[J]. 数据分析与知识发现, 2021, 5(2): 1-13.
[7] Chen Hao, Zhang Mengyi, Cheng Xiufeng. Identifying Cross-Region Patent Collaboration Opportunities Using LDA and Decision Trees——Case Study of Universities from Guangdong and Wuhan[J]. 数据分析与知识发现, 2021, 5(10): 37-50.
[8] Wei Tingxin,Bai Wenlei,Qu Weiguang. Sense Prediction for Chinese OOV Based on Word Embedding and Semantic Knowledge[J]. 数据分析与知识发现, 2020, 4(6): 109-117.
[9] Yu Chuanming,Yuan Sai,Zhu Xingyu,Lin Hongjun,Zhang Puliang,An Lu. Research on Deep Learning Based Topic Representation of Hot Events[J]. 数据分析与知识发现, 2020, 4(4): 1-14.
[10] Pan Youneng,Ni Xiuli. Recommending Online Medical Experts with Labeled-LDA Model[J]. 数据分析与知识发现, 2020, 4(4): 34-43.
[11] Su Chuandong,Huang Xiaoxi,Wang Rongbo,Chen Zhiqun,Mao Junyu,Zhu Jiaying,Pan Yuhao. Identifying Chinese / English Metaphors with Word Embedding and Recurrent Neural Network[J]. 数据分析与知识发现, 2020, 4(4): 91-99.
[12] Xu Jianmin,Zhang Liqing,Wang Miao. Tracking Static Topics with Bayesian Network[J]. 数据分析与知识发现, 2020, 4(2/3): 200-206.
[13] Wang Sili,Zhu Zhongming,Yang Heng,Liu Wei. Automatically Identifying Hypernym-Hyponym Relations of Domain Concepts with Patterns and Projection Learning[J]. 数据分析与知识发现, 2020, 4(11): 15-25.
[14] Chen Wenjie. Predicting Research Collaboration Based on Translation Model[J]. 数据分析与知识发现, 2020, 4(10): 28-36.
[15] Xinyu Zai,Xuedong Tian. Retrieving Scientific Documents with Formula Description Structure and Word Embedding[J]. 数据分析与知识发现, 2020, 4(1): 131-138.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn