结合内容和标签的Web文本聚类研究

doi:10.11925/infotech.1003-3513.2014.11.07

摘要
参考文献
相关文章
Metrics

全文: PDF(648 KB) HTML
输出: BibTeX | EndNote (RIS)

摘要

[目的] 探索社会标签与文本内容的结合对文本聚类的影响.[方法] 采用Engadget中英文博客数据, 使用TF×IDF、TextRank、TextRank×IDF三种特征抽取方法, 线性函数和Sigmod函数进行相似度加权, AP算法进行聚类.[结果] 结果表明, TF×IDF的聚类效果最好, 两种加权对英文博文聚类有不同程度的改善, 但在中文博文聚类中, Sigmod加权结果稍有下降, 线性加权比Sigmoid加权方法效果更好.[局限] 没有找出标签相似度与内容相似度最佳的权重系数.AP聚类算法不能应用于大数据, 聚簇过多影响聚类结果的展示.[结论] 社会标签与文本内容相似度的线性加权能改善Web文本聚类结果.

	服务

	把本文推荐给朋友
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	章成志
	顾晓雪

关键词 ：社会标签, 特征选择, 文本聚类

Abstract：

[Objective] This paper explores the infulence of the combination of social tagging and text content. [Methods] In this paper, taking the English and Chinese blogs for example, using TF×IDF, TextRank and TextRank×IDF as text feature extraction method, basing on tags combining with text content where two types weighted methods is used, and AP clustering algorithm is used to cluster samples. [Results] The results show that acts the best in the clustering of three feature extraction. And content weighted with tags improve different degree of the clustering of English blogs, but not for Chinese blogs in the method of Sigmoid. In two kinds of similarity weighted, linear method performs better than the Sigmoid method. [Limitations] The authors cannot find the best weight coefficient of tag similarity and content similarity. AP clustering algorithm can't apply to big data and a lot of clustering results interfered the visualization of show. [Conclusions] The weighted similarity of social tags and text content can improve the effect of the clutering of Web text.

Key words： Social tag Feature selection Text clustering

收稿日期: 2014-05-07

G250

基金资助:

本文系国家社会科学基金项目"在线社交网络中基于用户的知识组织模式研究"(项目编号:14BTQ033)和教育部人文社会科学基金规划项目"多语言高质量社会化标签生成及聚类研究"(项目编号:13YJA870020)的研究成果之一.

通讯作者: 章成志 E-mail: zhangcz@njust.edu.cn E-mail: zhangcz@njust.edu.cn

作者简介: 作者贡献声明: 顾晓雪: 设计研究方案, 实验设计与实施, 数据清洗与分析, 论文起草; 章成志: 提出研究思路, 设计研究方案, 数据采集及分析, 论文最终版本修订.

引用本文:

顾晓雪, 章成志. 结合内容和标签的Web文本聚类研究[J]. 现代图书情报技术, 2014, 30(11): 45-52.
Gu Xiaoxue, Zhang Chengzhi. Using Content and Tags for Web Text Clustering. New Technology of Library and Information Service, DOI：10.11925/infotech.1003-3513.2014.11.07.

链接本文:

http://manu44.magtech.com.cn/Jwk_infotech_wk3/CN/10.11925/infotech.1003-3513.2014.11.07

[1] Sebastiani F. Machine Learning in Automated Text Categorization [J]. ACM Computing Surveys, 2002, 34(1): 1-47.
[2] Salton G, Wong A, Yang C S. A Vector Space Model for Automatic Indexing [J]. Communications of the ACM, 1975, 18(11): 613-620.
[3] 马娜. 文本聚类研究[J]. 电脑知识与技术, 2009, 5(20): 5487-5489. (Ma Na. Research of Document Clustering [J]. Computer Knowledge and Technology, 2009, 5(20): 5487-5489.)
[4] Salton G, Yu C T. On the Construction of Effective Vocabularies for Information Retrieval [C]. In: Proceedings of the 1973 Meeting on Programming Languages and Information Retrieval. New York: ACM, 1973: 48-60.
[5] 吴夙慧, 成颖, 郑彦宁, 等. 文本聚类中文本表示和相似度计算研究综述[J]. 情报科学, 2012, 30(4): 622-627. (Wu Suhui, Cheng Ying, Zheng Yanning, et al. A Survey on Text Representation and Similarity Calculation in Text Clustering [J]. Information Science, 2012, 30(4): 622-627.)
[6] Manning C D, Raghavan P, Schütze H. Introduction to Information Retrieval [M]. Cambridge: Cambridge University Press, 2008.
[7] Luxburg U V. A Tutorial on Spectral Clustering[J]. Statistics and Computing, 2007, 17(4): 395-416.
[8] Trivedi A, Rai P, DuVall S L, et al. Exploiting Tag and Word Correlations for Improved Webpage Clustering [C]. In: Proceedings of the 2nd International Workshop on Search and Mining User-Generated Contents. New York: ACM, 2010: 3-12.
[9] Zhao Y, Karypis G, Fayyad U. Hierarchical Clustering Algorithms for Document Datasets [J]. Data Mining and Knowledge Discovery, 2005, 10(2): 141-168.
[10] Jing L, Ng M K, Xu J, et al. Subspace Clustering of Text Documents with Feature Weighting K-means Algorithm[C]. In: Proceedings of the 9th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining. Springer Berlin Heidelberg, 2005: 802-812.
[11] Kummamuru K, Dhawale A, Krishnapuram R. Fuzzy Co-Clustering of Documents and Keywords [C]. In: Proceedings of the 12th IEEE International Conference on Fuzzy Systems. 2003, 2: 772-777.
[12] 毛嘉莉. 基于K-means的文本聚类算法[J]. 计算机系统应用, 2009, 18(10): 85-87. (Mao Jiali. Text Clustering Algorithm Based on K-means [J]. Computer Systems & Applications, 2009, 18(10): 85-87.)
[13] 李星毅, 曾路平, 施化吉. 基于单词相似度的文本聚类[J]. 计算机工程与设计, 2009, 30(8): 1966-1968. (Li Xingyi, Zeng Luping, Shi Huaji. Text Clustering Based on Word Similarity [J]. Computer Engineering and Design, 2009, 30(8): 1966-1968.)
[14] 李云, 田素方, 李拓, 等. 基于概念格的 Web 文本聚类[J]. 计算机工程与应用, 2008, 44(23): 169-171. (Li Yun, Tian Sufang, Li Tuo, et al. Web Text Clustering Based on Concept Lattice [J]. Computer Engineering and Applications, 2008, 44(23): 169-171.)
[15] 何文静, 何琳. 基于社会标签的文本聚类研究[J]. 现代图书情报技术, 2013(7-8): 49-54. (He Wenjing, He Lin. Reasearch on Text Clustering Based on Social Tagging [J]. New Technology of Library and Information Service, 2013 (7-8): 49-54.)
[16] 杨鲲, 马慧芳, 史忠植. 基于社会标注的 Web 资源语义聚类研究[J]. 高技术通讯, 2012, 22(1): 48-54. (Yang Kun, Ma Huifang, Shi Zhongzhi. Semantic Clustering of Web Resources Based on Social Annotation [J]. Chinese High Technology Letters, 2012, 22(1): 48-54.)
[17] Li P, Wang B, Jin W. Improving Web Document Clustering Through Employing User-Related Tag Expansion Techniques [J]. Journal of Computer Science and Technology, 2012, 27(3): 554-566.
[18] 贺秋芳, 曾启杰, 蔡延光. 挖掘用户标签的增强型社区网页聚类算法[J]. 微电子学与计算机, 2013, 30(2): 74-77. (He Qiufang, Zeng Qijie, Cai Yan'guang. Enhanced Social Web Clustering Algorithm of Mining Information [J]. Microelectronics & Computer, 2013, 30(2): 74-77.)
[19] Lu C, Chen X, Park E K. Exploit the Tripartite Network of Social Tagging for Web Clustering [C]. In: Proceedings of the 18th ACM Conference on Information and Knowledge Management. New York: ACM, 2009: 1545-1548.
[20] 叶宇飞, 安世全, 代劲. 一种新的Web中文文本聚类方法研究[J]. 计算机应用与软件, 2013, 30(12): 222-225, 287. (Ye Yufei, An Shiquan, Dai Jin. Research on a Novel Web Chinese Text Clustering Method [J]. Computer Applications and Software, 2013, 30(12): 222-225, 287.)
[21] Ramage D, Heymann P, Manning C D, et al. Clustering the Tagged Web [C]. In: Proceedings of the 2nd ACM International Conference on Web Search and Data Mining. New York: ACM, 2009: 54-63.
[22] 李鹏, 王斌, 石志伟, 等. Tag-Text Rank: 一种基于Tag的网页关键词抽取方法[J]. 计算机研究与发展, 2012, 49(11): 2344-2351. (Li Peng, Wang Bin, Shi Zhiwei, et al. Tag-Text Rank: A Webpage Keyword Extraction Method Based on Tags [J]. Journal of Computer Research and Development, 2012, 49(11): 2344-2351.)
[23] 姚清耘. 基于向量空间模型的中文文本聚类方法的研究[D]. 上海: 上海交通大学, 2008. (Yao Qingyun. Research of VSM-Based Chinese Text Clustering Algorithms [D]. Shanghai: Shanghai Jiaotong University, 2008.)
[24] Mihalcea R, Tarau P. Text Rank: Bringing Order into Texts [C]. In: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, Barcelona, Spain. 2004.
[25] Ehrig M, Staab S. QOM–quick Ontology Mapping [C]. In: Proceedings of the 3rd International Semantic Web Conference (ISWC 2004). Springer Berlin Heidelberg, 2004: 683-697.
[26] Peukert E, Massmann S, Koenig K. Comparing Similarity Combination Methods for Schema Matching [C]. In: Proceedings of GI Jahrestagung (1). 2010: 692-701.
[27] 何琳. 基于多策略的领域本体术语抽取研究[J]. 情报学报, 2012, 31(8): 798-804. (He Lin. Domain Ontology Terminology Extraction Based on Integrated Strategy Method [J]. Journal of the China Society for Scientific and Technical Information, 2012, 31(8): 798-804.)
[28] Frey B J, Dueck D. Clustering by Passing Messages Between Data Points [J]. Science, 2007, 315(5814): 972-976.
[29] Tan P N, Steinbach M, Kumar V. 数据挖掘导论[M]. 范明, 范宏建译. 北京: 人民邮电出版社, 2006: 340-341. (Tan P N, Steinbach M, Kumar V. Introduction to Data Mining [M]. Translated by Fan Ming, Fan Hongjian. Beijing: Posts & Telecom Press, 2006: 340-341.)

[1]	周成,魏红芹. *专利价值评估与分类研究^——基于自组织映射支持向量机**[J]. 数据分析与知识发现, 2019, 3(5): 117-124.
[2]	梁家铭,赵洁,Jianlong Zhou,董振宁. 用户隐式行为挖掘在抗信誉共谋中的应用研究^*[J]. 数据分析与知识发现, 2019, 3(5): 125-138.
[3]	陆泉,朱安琪,张霁月,陈静. *中文网络健康社区中的用户信息需求挖掘研究^——以求医网肿瘤板块数据为例**[J]. 数据分析与知识发现, 2019, 3(4): 22-32.
[4]	温廷新,李洋子,孙静霜. 基于多因素特征选择与AFOA/K-means的新闻热点发现方法^*[J]. 数据分析与知识发现, 2019, 3(4): 97-106.
[5]	谭章禄,王兆刚,胡翰. 一种基于χ²统计的特征分类选择方法研究^*[J]. 数据分析与知识发现, 2019, 3(2): 72-78.
[6]	张涛,马海群. 一种基于LDA主题模型的政策文本聚类方法研究^*[J]. 数据分析与知识发现, 2018, 2(9): 59-65.
[7]	温廷新,李洋子,孙静霜. 基于改进的果蝇优化算法的文本特征选择优化模型[J]. 数据分析与知识发现, 2018, 2(5): 59-69.
[8]	操玮,李灿,贺婷婷,朱卫东. 基于集成学习的中国P2P网络借贷信用风险预警模型的对比研究^*[J]. 数据分析与知识发现, 2018, 2(10): 65-76.
[9]	李志鹏,李卫忠. 基于可拓小生境量子粒子群算法的特征选择^*[J]. 数据分析与知识发现, 2017, 1(7): 82-89.
[10]	官琴, 邓三鸿, 王昊. 中文文本聚类常用停用词表对比研究^*[J]. 数据分析与知识发现, 2017, 1(3): 72-80.
[11]	张越,王东波,朱丹浩. 面向食品安全突发事件汉语分词的特征选择及模型优化研究^*[J]. 数据分析与知识发现, 2017, 1(2): 64-72.
[12]	李湘东,阮涛,刘康. 基于维基百科的多种类型文献自动分类研究^*[J]. 数据分析与知识发现, 2017, 1(10): 43-52.
[13]	路永和,陈景煌. 混合蛙跳算法在文本分类特征选择优化中的应用^*[J]. 数据分析与知识发现, 2017, 1(1): 91-101.
[14]	孟园,王洪伟. 基于文本内容特征选择的评论质量检测^*[J]. 现代图书情报技术, 2016, 32(4): 40-47.
[15]	陈东沂,周子程,蒋盛益,王连喜,吴佳林. 面向企业微博的客户细分框架^*[J]. 现代图书情报技术, 2016, 32(2): 43-51.

Viewed

Full text

Abstract

Cited

Shared

Discussed