Please wait a minute...
Data Analysis and Knowledge Discovery  2018, Vol. 2 Issue (12): 68-76    DOI: 10.11925/infotech.2096-3467.2018.0391
Current Issue | Archive | Adv Search |
Classifying Chinese Texts with CapsNet
Guoming Feng,Xiaodong Zhang(),Suhui Liu
School of Economics and Management, University of Science and Technology Beijing, Beijing 100083, China
Download: PDF(732 KB)   HTML ( 1
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This study tries to address the issues facing long text representation and use CapsNet to improve the accuracy of Chinese text classification. [Methods] First, we proposed a LDA matrix and word vector to represent the long texts. Then, we constructed a Chinese classification model based on CapsNet. Third, we examined the proposed model with Sogou news corpus and the text classification corpus of Fudan University. Finally, we compared our results with those of the classic models (e.g., TextCNN, DNN and so on). [Results] The performance of CapsNet model was better than other models. The classification accuracy in five categories of short and long texts reached 89.6% and 96.9% respectively. The convergence speed of the proposed model was almost two times faster than that of the CNN model. [Limitations] The computational complexity of the model is high, which limits the size of testing corpus. [Conclusions] The proposed Chinese text representation method and the modified CapsNet model have better accuracy, convergence speed and robustness than the existing ones.

Key wordsText Categorization      CapsNet      Deep Learning      Text Representation      TextCNN     
Received: 08 April 2018      Published: 16 January 2019

Cite this article:

Guoming Feng,Xiaodong Zhang,Suhui Liu. Classifying Chinese Texts with CapsNet. Data Analysis and Knowledge Discovery, 2018, 2(12): 68-76.

URL:

http://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2018.0391     OR     http://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2018/V2/I12/68

[1] 唐明, 朱磊, 邹显春. 基于Word2Vec的一种文档向量表示[J]. 计算机科学, 2016, 43(6): 214-217.
[1] (Tang Ming, Zhu Lei, Zou Xianchun.Document Vector Representation Based on Word2Vec[J]. Computer Science, 2016, 43(6): 214-217.)
[2] 幸凯. 基于卷积神经网络的文本表示建模方法研究[D]. 武汉: 华中师范大学, 2017.
[2] (Xing Kai.Research on Text Modeling Based on Convolutional Neural Network Approaches[D]. Wuhan: Central China Normal University, 2017.)
[3] 黄磊, 杜昌顺. 基于递归神经网络的文本分类研究[J]. 北京化工大学学报: 自然科学版, 2017, 44(1): 98-104.
[3] (Huang Lei, Du Changshun.Application of Recurrent Neural Networks in Text Classification[J]. Journal of Beijing University of Chemical Technology: Natural Science Edition, 2017, 44(1): 98-104.)
[4] Sabour S, Frosst N, Hinton G E.Dynamic Routing Between Capsules[OL]. arXiv Preprint. arXiv: 1710.09829.
[5] Salton G, Wong A, Yang C S.A Vector Space Model for Automatic Indexing[J]. Communications of the ACM,1975, 18(11): 613-620.
[6] Deerwester S, Dumais S, Furnas G W, et al.Indexing by Latent Semantic Analysis[J]. Journal of the American Society for Information Science, 1990, 41(6): 391-407.
[7] Hofmann T.Unsupervised Learning by Probabilistic Latent Semantic Analysis[J]. Machine Learning, 2001, 42(1-2): 177-196.
[8] Blei D M, Ng A Y, Jordan M I.Latent Dirichlet Allocation[J]. Journal of Machine Learning Research, 2003, 3(2): 993-1022.
[9] Mikolov T, Chen K, Corrado G, et al.Efficient Estimation of Word Representations in Vector Space[OL]. arXiv Preprint. arXiv: 1301.3781.
[10] Joachims T.Text Categorization with Support Vector Machines: Learning with Many Relevant Features[C]// Proceedings of the 10th European Conference on Machine Learning. 1998: 137-142.
[11] Kim Y.Convolutional Neural Networks for Sentence Classification[OL]. arXiv Preprint. arXiv: 1408.5882.
[12] Kalchbrenner N, Grefenstette E, Blunsom P.A Convolutional Neural Network for Modelling Sentences[OL]. arXiv Preprint. arXiv: 1404.2188.
[13] Liu P, Qiu X, Huang X.Recurrent Neural Network for Text Classification with Multi-Task Learning[C]// Proceedings of the 25th International Joint Conference on Artificial Intelligence. 2016: 2873-2879.
[14] Joulin A, Grave E, Bojanowski P, et al.Bag of Tricks for Efficient Text Classification[C]// Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. 2016: 427-431.
[15] 崔建明, 刘建明, 廖周宇. 基于SVM算法的文本分类技术研究[J]. 计算机仿真, 2013, 30(2): 299-302.
[15] (Cui Jianming, Liu Jianming, Liao Zhouyu.Research of Text Categorization Based on Support Vector Machine[J]. Computer Simulation, 2013, 30(2): 299-302.)
[16] 李玉鑑, 王影, 冷强奎. 基于最近邻子空间搜索的两类文本分类方法[J]. 计算机工程与科学, 2015, 37(1): 168-172.
[16] (Li Yujian, Wang Ying, Leng Qiangkui.Two-class Text Categorization Using Nearest Subspace Search[J]. Computer Engineering and Science, 2015, 37(1): 168-172.)
[17] 吕超镇, 姬东鸿, 吴飞飞. 基于LDA特征扩展的短文本分类[J]. 计算机工程与应用, 2015, 51(4):123-127.
[17] (Lv Chaozhen, Ji Donghong, Wu Feifei.Short Text Classification Based on LDA Feature Extension[J]. Computer Engineering and Applications, 2015, 51(4): 123-127.)
[18] 郭东亮, 刘小明, 郑秋生. 基于卷积神经网络的互联网短文本分类方法[J]. 计算机与现代化, 2017(4): 78-81.
[18] (Guo Dongliang, Liu Xiaoming, Zheng Qiusheng.Internet Short-text Classification Method Based on CNNs[J]. Computer and Modernization, 2017(4): 78-81.)
[19] 陈杰, 陈彩, 梁毅. 基于Word2Vec的文档分类方法[J]. 计算机系统应用, 2017, 26(11): 159-164.
[19] (Chen Jie, Chen Cai, Liang Yi.Document Classification Method Based on Word2Vec[J]. Computer Systems & Applications, 2017, 26(11): 159-164.)
[20] 夏从零, 钱涛, 姬东鸿. 基于事件卷积特征的新闻文本分类[J]. 计算机应用研究, 2017, 34(4): 991-994.
[20] (Xia Congling, Qian Tao, Ji Donghong.Event Convolutional Feature Based News Documents Classification[J]. Application Research of Computers, 2017, 34(4): 991-994.)
[1] Mengji Zhang,Wanyu Du,Nan Zheng. Predicting Stock Trends Based on News Events[J]. 数据分析与知识发现, 2019, 3(5): 11-18.
[2] Jingjing Pei,Xiaoqiu Le. Identifying Coordinate Text Blocks in Discourses[J]. 数据分析与知识发现, 2019, 3(5): 51-56.
[3] Zhanglu Tan,Zhaogang Wang,Han Hu. Study on a Method of Feature Classification Selection Based on χ2 Statistics[J]. 数据分析与知识发现, 2019, 3(2): 72-78.
[4] Li Yu,Li Qian,Changlei Fu,Huaming Zhao. Extracting Fine-grained Knowledge Units from Texts with Deep Learning[J]. 数据分析与知识发现, 2019, 3(1): 38-45.
[5] Changlei Fu,Li Qian,Huaping Zhang,Huaming Zhao,Jing Xie. Mining Innovative Topics Based on Deep Learning[J]. 数据分析与知识发现, 2019, 3(1): 46-54.
[6] Bengong Yu,Peihang Zhang,Qingtang Xu. Selecting Products Based on F-BiGRU Sentiment Analysis[J]. 数据分析与知识发现, 2018, 2(9): 22-30.
[7] Xiangdong Li,Fan Gao,Youhai Li. Categorizing Documents Automatically within Common Semantic Space[J]. 数据分析与知识发现, 2018, 2(9): 66-73.
[8] Wei Lu,Mengqi Luo,Heng Ding,Xin Li. Image Annotation Tags by Deep Learning and Real Users: A Comparative Study[J]. 数据分析与知识发现, 2018, 2(5): 1-10.
[9] Yanhui Xiao,Xin Wang,Wen’gang Feng,Huawei Tian,Shaozhong Wu,Lihua Li. Predicting Crime Locations Based on Long Short Term Memory and Convolutional Neural Networks[J]. 数据分析与知识发现, 2018, 2(10): 15-20.
[10] Wengang Feng,Jing Huang. Early Warning for Civil Aviation Security Checks Based on Deep Learning[J]. 数据分析与知识发现, 2018, 2(10): 46-53.
[11] Jiaheng Hu,Yonghua Cen,Chengyao Wu. Constructing Sentiment Dictionary with Deep Learning: Case Study of Financial Data[J]. 数据分析与知识发现, 2018, 2(10): 95-102.
[12] Sanhong Deng,Yuyangzi Fu,Hao Wang. Multi-Label Classification of Chinese Books with LSTM Model[J]. 数据分析与知识发现, 2017, 1(7): 52-60.
[13] Danhao Zhu, Lei Yang, Dongbo Wang. Recognizing Chinese Organization Names Based on Deep Learning: A Recurrent Network Model[J]. 数据分析与知识发现, 2016, 32(12): 36-43.
[14] Liyi Zhang,Chang Liu. Combine Deep Belief Networks and Fuzzy Set for Recognition of Fraud Transaction[J]. 现代图书情报技术, 2016, 32(1): 32-39.
[15] Xu Dongdong, Wu Shaobo. An Improved TF-IDF Feature Selection Based on Categorical Description[J]. 现代图书情报技术, 2015, 31(3): 39-48.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn