[Objective] Thisf paper aims to improve the prediction of stock prices with the help of investors’sentimental characteristics. [Methods] First, we constructed investors’ sentimental characteristics with the RoBERTa model and extracted the stock price characteristics with the TCN network. Then, we used the attention mechanism to merge these characteristics. Finally, we constructed the new RoBERTa-TCN model for stock price prediction. [Results] Compared with the experimental results of three models LSTM, GRU and TCN on six stock datasets, RoBERTa-TCN model has an average improvement of about 0.4906 on four different evaluation indicators. [Limitations] We did not examine the impacts of trading dates on the stock prices. [Conclusions] The RoBERTa-TCN model could effectively predict stock prices.
严冬梅, 何雯馨, 陈智. 融合情感特征的基于RoBERTa-TCN的股价预测研究[J]. 数据分析与知识发现, 2022, 6(12): 123-134.
Yan Dongmei, He Wenxin, Chen Zhi. Predicting Stock Prices Based on RoBERTa-TCN and Sentimental Characteristics. Data Analysis and Knowledge Discovery, 2022, 6(12): 123-134.
Hochreiter S, Schmidhuber J. Long Short-Term Memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
doi: 10.1162/neco.1997.9.8.1735
pmid: 9377276
[2]
Cho K, van Merrienboer B, Gulcehre C, et al. Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014: 1724-1734.
[3]
Bai S J, Kolter J Z, Koltun V. An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling[OL]. arXiv Preprint, arXiv: 1803.01271.
(Zhao Cheng, Ye Yaowei, Yao Minghai. Stock Volatility Forecast Based on Financial Text Emotion[J]. Computer Science, 2020, 47(5): 79-83.)
doi: 10.11896/jsjkx.190400145
[5]
Chen K, Zhou Y, Dai F Y. A LSTM-Based Method for Stock Returns Prediction: A Case Study of China Stock Market[C]// Proceedings of the 2015 IEEE International Conference on Big Data. 2015: 2823-2824.
[6]
Fischer T, Krauss C. Deep Learning with Long Short-Term Memory Networks for Financial Market Predictions[J]. European Journal of Operational Research, 2018, 270(2): 654-669.
doi: 10.1016/j.ejor.2017.11.054
(Pei Dawei, Zhu Ming. Stock Price Prediction Based on Multiple-Factor and Multi-Variable Long Short Term Memory[J]. Computer Systems & Applications, 2019, 28(8): 30-38.)
[8]
Kim T, Kim H Y. Forecasting Stock Prices with a Feature Fusion LSTM-CNN Model Using Different Representations of the Same Data[J]. PLoS One, 2019, 14(2): e0212320.
(Zhang Qianyu, Yan Dongmei, Han Jiatong. Research on Stock Price Prediction Combined with Deep Learning and Decomposition Algorithm[J]. Computer Engineering and Applications, 2021, 57(5): 56-64.)
doi: 10.3778/j.issn.1002-8331.2006-0444
(Chen Yufan, Wen Mi, Zhang Kai, et al. Short-Term Photovoltaic Output Forecasting Based on Similar Day Matching and TCN-Attention[J]. Electrical Measurement & Instrumentation, 2022, 59(10): 108-116.)
(Yang Juan. Empirical Analysis of the Impact of Internet Financial News on Stock: Analysis of Company News[D]. Chengdu: Southwestern University of Finance and Economics, 2012.)
(Wang Ting, Yang Wenzhong. Review of Text Sentiment Analysis Methods[J]. Computer Engineering and Applications, 2021, 57(12): 11-24.)
doi: 10.3778/j.issn.1002-8331.2101-0022
[14]
Ren R, Wu D D, Liu T X. Forecasting Stock Market Movement Direction Using Sentiment Analysis and Support Vector Machine[J]. IEEE Systems Journal, 2019, 13(1): 760-770.
doi: 10.1109/JSYST.2018.2794462
[15]
Dong J H. Financial Investor Sentiment Analysis Based on FPGA and Convolutional Neural Network[J]. Microprocessors and Microsystems, 2020: 103418.
[16]
Devlin J, Chang M W, Lee Kenton, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[OL]. arXiv Preprint, arXiv: 1810.04805.
[17]
Munikar M, Shakya S, Shrestha A. Fine-Grained Sentiment Classification Using BERT[C]// Proceedings of 2019 Artificial Intelligence for Transforming Business and Society. 2019: 1-5.
[18]
Liu Y H, Ott M, Goyal N, et al. BoBERTa: A Robustly Optimized BERT Pretraining Approach[OL]. arXiv Preprint, arXiv: 1907. 11692.
[19]
Kaminski J. Nowcasting the Bitcoin Market with Twitter Signals[OL]. arXiv Preprint, arXiv: 1406.7577.
(Fan Pengying, Yang Yin, Zhang Zhengping, et al. The Relationship Between Individual Stock Investor Sentiment and the Stock Yield——Based on the Perspective of Stock Evaluation Information[J]. Mathematics in Practice and Theory, 2021, 51(16): 305-320.)
(Cai Qihang. Analysis of the Influence of Media Sentiment on Stock Returns——An Extended Research Based on Natural Language Processing and Multi-Factor Models[D]. Hangzhou: Zhejiang University, 2021.)
[22]
Jing N, Wu Z, Wang H F. A Hybrid Model Integrating Deep Learning with Investor Sentiment Analysis for Stock Price Prediction[J]. Expert Systems with Applications, 2021, 178: 115019.
doi: 10.1016/j.eswa.2021.115019
[23]
Vargas M R, dos Anjos C E M, Bichara G L G, et al. Deep Learning for Stock Market Prediction Using Technical Indicators and Financial News Articles[C]// Proceedings of 2018 International Joint Conference on Neural Networks. 2018: 1-8.
(Xu Yuemei, Wang Zihou, Wu Zixin. Predicting Stock Trends with CNN-BiLSTM Based Multi-Feature Integration Model[J]. Data Analysis and Knowledge Discovery, 2021, 5(7): 126-137.)
[25]
Li X D, Wu P J, Wang W P. Incorporating Stock Prices and News Sentiments for Stock Market Prediction: A Case of Hong Kong[J]. Information Processing & Management, 2020, 57(5): 102212.
doi: 10.1016/j.ipm.2020.102212
[26]
Kanavos A, Vonitsanos G, Mohasseb A, et al. An Entropy-Based Evaluation for Sentiment Analysis of Stock Market Prices Using Twitter Data[C]// Proceedings of 2020 15th International Workshop on Semantic and Social Media Adaptation and Personalization. 2020: 1-7.
[27]
Ruder S, Peters M E, Swayamdipta S, et al. Transfer Learning in Natural Language Processing[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Tutorials. 2019: 15-18.
[28]
Araci D. FinBERT: Financial Sentiment Analysis with Pre-trained Language Models[OL]. arXiv Preprint, arXiv: 1908.10063.
[29]
Li M G, Li W R, Wang F, et al. Applying BERT to Analyze Investor Sentiment in Stock Market[J]. Neural Computing and Applications, 2021, 33(10): 4663-4676.
doi: 10.1007/s00521-020-05411-7
[30]
Li M Z, Chen L, Zhao J, et al. Sentiment Analysis of Chinese Stock Reviews Based on BERT Model[J]. Applied Intelligence, 2021, 51(7): 5016-5024.
doi: 10.1007/s10489-020-02101-8
[31]
van den Oord A, Dieleman S, Zen H G, et al. WaveNet: A Generative Model for Raw Audio[OL]. arXiv Preprint, arXiv: 1609.03499.
[32]
He K M, Zhang X Y, Ren S Q, et al. Deep Residual Learning for Image Recognition[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016: 770-778.
[33]
杨浩东. 基于深度注意力机制的视频中人体动作识别[D]. 长沙: 国防科技大学, 2018.
[33]
(Yang Haodong. Attention Mechanism Based Deep Network for Human Action Recognition in Video[D]. Changsha: National University of Defense Technology, 2018.)
[34]
Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017: 6000-6010.
[35]
Jin Z G, Yang Y, Liu Y H. Stock Closing Price Prediction Based on Sentiment Analysis and LSTM[J]. Neural Computing and Applications, 2020, 32(13): 9713-9729.
doi: 10.1007/s00521-019-04504-2
