[Objective] This study aims to systematically reveal the principles of the traditional deep representation models and the latest pre-training models and explore their differences in text mining tasks. [Methods] Using the comparative study method, we investigate the differences between the traditional models and the latest models on the six datasets of CR, MR, MPQA, Subj, SST-2 and TREC from the model side and the experimental side. [Results] On the six tasks, the XLNet model achieved the best average F1 value (0.9186) overall, which is better than ELMo (0.8090), BERT(0.8983), Word2Vec(0.7692), GloVe(0.7576) and FastText(0.7506). [Limitations] Due to space limitations, the empirical research focuses on classification tasks of text mining. It has not compared the effects of multiple vocabulary representation methods in machine translation, Q&A and other tasks. [Conclusions] The traditional deep representation learning model and the latest pre-training model still have large effect differences in text mining tasks.