Computing Text Similarity Based on Concept Vector Space
Li Lin1, Li Hui2()
1School of Foreign Studies, Anhui University, Hefei 230601, China 2Department of Electronics Engineering and Information Science, University of Science and Technology of China, Hefei 230027, China
[Objective] This paper proposes a method to compute the semantic similarity of texts based on a concept vector space model. [Methods] First, we analyzed the text by dependency parser and extracted key words. Then, we used word embedding method to build vector space for each document. Third, we measured similarities between the two vector spaces and actual texts. Finally, we evaluated the new similarity measures with the data set of short texts and proposed an algorithm for long document classification. [Results] The proposed method effectively measured the semantic similarity of short texts and long documents. The accuracy of document classification was over 92% for the long ones. [Limitations] The performance of our method relies on the quality of dependency parser and word embedding vectors. [Conclusions] Combining linguistics theory and word embedding technique could efectively measure the semantic similarity among texts. This new method also reduces computation complexity and could be used in document classification, text clustering, and automatic question answering systems.
李琳, 李辉. 一种基于概念向量空间的文本相似度计算方法[J]. 数据分析与知识发现, 2018, 2(5): 48-58.
Li Lin,Li Hui. Computing Text Similarity Based on Concept Vector Space. Data Analysis and Knowledge Discovery, 2018, 2(5): 48-58.
The dollar has hit its highest level against the euro in almost three months after the Federal Reserve head said the US trade deficit is set to stabilise.
(Chen Erjing, Jiang Enbo.Review of Studies on Text Similarity Measures[J]. Data Analysis and Knowledge Discovery, 2017, 1(6): 1-11.)
[2]
Salton G, Wong A, Yang C S.A Vector Space Model for Automatic Indexing[J]. Communications of the ACM, 1975, 18(11): 613-620.
doi: 10.1145/361219.361220
[3]
Salton G, Buckley C.Term-Weighting Approaches in Automatic Text Retrieval[J]. Information Processing & Management, 1988, 24(5): 513-523.
[4]
Landauer T K, Foltz P W, Laham D.An Introduction to Latent Semantic Analysis[J]. Discourse Processes, 1998, 25(2-3): 259-284.
doi: 10.1080/01638539809545028
[5]
Hofmann T.Probabilistic Latent Semantic Analysis[C]// Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence. 1999: 289-296.
[6]
Blei D M, Ng A Y, Jordan M I.Latent Dirichlet Allocation[J]. Journal of Machine Learning Research, 2003, 3: 993-1022.
[7]
Miller G A.WordNet: A Lexical Database for English[J]. Communications of the ACM, 1995, 38(11): 39-41.
(Dong Zhendong, Dong Qiang, Hao Changling.Theoretical Findings of HowNet[J]. Journal of Chinese Information Processing, 2007, 21(4): 3-9.)
[9]
梅家驹, 竺一鸣, 高蕴琦, 等. 同义词词林[M]. 上海: 上海辞书出版社, 1983.
[9]
(Mei Jiaju, Zhu Yiming, Gao Yunqi, et al.Tongyici Cilin [M]. Shanghai: Shanghai Lexicographical Publishing House, 1983.)
[10]
Pedersen T, Patwardhan S, Michelizzi J.WordNet: Similarity - Measuring the Relatedness of Concepts[C]// Proceedings of the 19th National Conference on Artificial Intelligence. 2004: 38-41.
(Jiang Min, Xiao Shibin, Wang Hongwei, et al.An Improved Word Similarity Computing Method Based on HowNet[J]. Journal of Chinese Information Processing, 2008, 22(5): 84-89.)
(Tian Jiule, Zhao Wei.Words Similarity Algorithm Based on Tongyici Cilin in Semantic Web Adaptive Learning System[J]. Journal of Jilin University: Information Science Edition, 2010, 28(6):602-608.)
doi: 10.3969/j.issn.1671-5896.2010.06.011
[13]
Gabrilovich E, Markovitch S.Computing Semantic Relatedness Using Wikipedia-Based Explicit Semantic Analysis[C]// Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI’07). 2007: 1606-1611.
(Zhan Zhijian, Liang Lina, Yang Xiaoping.Word Similarity Measurement Based on BaiduBaike[J]. Computer Science, 2013, 40(6): 199-202.)
doi: 10.3969/j.issn.1002-137X.2013.06.043
[16]
Mikolov T, Sutskever I, Chen K, et al.Distributed Representations of Words and Phrases and Their Compositionality[C]// Advances in Neural Information Processing Systems (NIPS 2013). 2013: 3111-3119.
[17]
Shao Y.HCTI at SemEval-2017 Task 1: Use Convolutional Neural Network to Evaluate Semantic Textual Similarity[C]// Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017). 2017: 130-133.
[18]
Tai K S, Socher R, Manning C D.Improved Semantic Representations from Tree-Structured Long Short-Term Memory Networks[C]// Proceedings of the 53rd Annual Meetings of Association for Computational Linguistics. 2015: 1556-1566.
[19]
Kim H K, Kim H, Cho S.Bag-of-Concepts: Comprehending Document Representation Through Clustering Words in Distributed Representation[J]. Neurocomputing, 2017, 266: 366-352.
(Li Feng, Hou Jiaying, Zeng Rongren, et al.Research on Multi-feature Sentence Similarity Computing Method with Word Embedding[J]. Journal of Frontiers of Computer Science and Technology, 2017, 11(4): 608-618.)
doi: 10.3778/j.issn.1673-9418.1604029
(Li Xiao, Xie Hui, Li Lijie.Research on Sentence Semantic Similarity Calculation Based on Word2Vec[J]. Computer Science, 2017, 44(9): 256-260.)
doi: 10.11896/j.issn.1002-137X.2017.09.048
[22]
刘海涛. 依存语法的理论与实践[M]. 北京: 科学出版社, 2009.
[22]
(Liu Haitao.Dependency Grammar from Theory to Practice[M]. Beijing: Science Press, 2009.)
[23]
Choi J D, Palmer M.Guidelines for the Clear Style Constituent to Dependency Conversion [R]. University of Colorado Boulder, 2012.
[24]
Cer D, Diab M, Agirre E, et al.SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation[C]// Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017). 2017: 1-14.
[25]
Mikolov T, Chen K, Corrado G, et al. Efficient Estimation of Word Representations in Vector Space[OL]. arXiv Preprint, arXiv:1301.3781, 2013.
[26]
Le Q, Mikolov T.Distributed Representations of Sentences and Documents[C]// Proceedings of the 31st International Conference on Machine Learning (ICML-14). 2014: 1188-1196.
[27]
Kusner M, Sun Y, Kolkin N, et al.From Word Embeddings to Document Distances[C]// Proceedings of the 32nd International Conference on Machine Learning. 2015: 957-966.
[28]
Maaten L, Hinton G.Visualizing Data Using t-SNE[J]. Journal of Machine Learning Research, 2008, 9: 2579-2605.
doi: 10.1007/s10846-008-9235-4