Method for Automatically Generating Online Comments
Liu Xinran1,2,Xu Yabin1,2(),Li Jixian3
1Beijing Key Laboratory of Network Culture and Digital Communication, Beijing University of Information Science and Technology, Beijing 100101, China 2School of Computer Science, Beijing University of Information Science and Technology, Beijing 100101, China 3School of Humanities and Education, Beijing Open University, Beijing 100081, China
[Objective] This paper proposes a Temporal Sequence Generative Adversarial Network (T-SeqGAN) automatically generating online comments, aiming to counteract malicious information on social networks and guide the correct direction of public opinion. [Methods] First, we modified the Sequence Generative Adversarial Network (SeqGAN) generator to a Seq2Seq structure. Then, we used the bidirectional gated recurrent unit (BiGRU) and the sequential convolutional neural network (TCN) as the skeleton network of the encoder and decoder, respectively. Next, we improved the similarity of the syntactic structure and semantic features between the generated posts and the real online comments. Finally, we modified the discriminator of SeqGAN to a model combing TCN and attention mechanism layers to improve the fluency of generated posts. [Results] Compared with the baseline model, the comments generated by the proposed model have significantly higher BLEU-2 (0.799 35), BLEU-3(0.603 96), BLEU-4(0.476 42), and KenLM (-27.670 29)metrics, as well as lower PPL(0.752 47) metrics. [Limitations] The vocabulary and language style of the generated posts are limited by actual posts, and the applicability of our method is limited. [Conclusions] The comments generated by the proposed model have higher syntactic and grammatical correctness and higher similarity to the real-world ones, which can guide the correct direction of public opinion on social networks.
Cheng Xinbin. Analysis and Countermeasures of Public Opinion Guidance Research on Major Public Opinions and Emergencies[J]. Journal of Southwest Minzu University (Humanities and Social Science), 2022, 43(2): 235-240.)
Yu Xiaoqing. The Problems and Countermeasures of Coping Mechanism on Ideological Internet Public Opinion[J]. Journal of Nanjing University of Posts and Telecommunications (Social Science Edition), 2021, 23(5): 37-47.)
[3]
Zou X H, Lin C, Zhang Y J, et al. To be an Artist: Automatic Generation on Food Image Aesthetic Captioning[C]// Proceedings of the 32nd International Conference on Tools with Artificial Intelligence. IEEE, 2020: 779-786.
[4]
Zeng W H, Abuduweili A, Li L, et al. Automatic Generation of Personalized Comment Based on User Profile[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics:Student Research Workshop. ACL, 2019: 229-235.
[5]
Li W, Xu J J, He Y C, et al. Coherent Comments Generation for Chinese Articles with a Graph-to-Sequence Model[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. ACL, 2019: 4843-4852.
[6]
Casar Morejon C. Automatic Generation of Comments on Twitter Based on News[D]. Universität Politècnica de Catalunya, 2018.
(Ruan Yeli. The Research of Text Generation with Considering External Knowledge and Emotional Attributes Based on GAN[D]. Wuhan: Zhongnan University of Economics and Law, 2020.)
[8]
Kingma D P, Welling M. Auto-Encoding Variational Bayes[OL]. arXiv Preprint, arXiv: 1312.6114.
[9]
Li C Y, Gao X, Li Y, et al. Optimus: Organizing Sentences via Pre-Trained Modeling of a Latent Space[C]// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 2020: 4678-4699.
[10]
胡盛伟. 基于深度学习的条件文本生成技术与应用研究[D]. 泉州: 华侨大学, 2020.
[10]
(Hu Shengwei. Research on Conditional Text Generation Technology and Its Application Based on Deep Learning[D]. Quanzhou: Huaqiao University, 2020.)
[11]
Liu D Y, Liu G S. A Transformer-Based Variational Autoencoder for Sentence Generation[C]// Proceedings of the 2019 International Joint Conference on Neural Networks. IEEE, 2019: 1-7.
[12]
张超. 生成对抗网络在文本生成中的应用研究[D]. 武汉: 湖北工业大学, 2019.
[12]
(Zhang Chao. Application Research of Generative Adversarial Networks in Text Generation[D]. Wuhan: Hubei University of Technology, 2019.)
(Liu Lei. Research on Text Sentiment Classfication Based on Generative Adversarial Network and Heterogenous Ensemble Learning[D]. Nanjing: Nanjing University of Posts and Telecommunications, 2020.)
[14]
Yu L T, Zhang W N, Wang J, et al. SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient[C]// Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2017.
[15]
王姿雯. 基于深度学习的多条件个性化文本生成[D]. 北京: 北京邮电大学, 2019.
[15]
(Wang Ziwen. Multi-Conditional Generation of Personalized Texts Based on Deep Learning[D]. Beijing: Beijing University of Posts and Telecommunications, 2019.)
(Yan Dan, He Jun, Liu Hongyan, et al. Considering Grade Information for Music Comment Text Automatic Generation[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(8): 1389-1396.)
doi: 10.3778/j.issn.1673-9418.1908075
(Han Hu, Sun Tianyue, Zhao Qitao. Generative Adversarial Networks with Autoencoder for Text Generation[J]. Computer Engineering & Science, 2020, 42(9): 1704-1710.)
[18]
Zheng H T, Wang W, Chen W, et al. Automatic Generation of News Comments Based on Gated Attention Neural Networks[J]. IEEE Access, 2017, 6: 702-710.
doi: 10.1109/ACCESS.2017.2774839
(Zhu Xiangqi, Zhang Zhonglin, Li Linchuan, et al. Short Text Classification Based on Improved Part of Speech Information and ACBiLSTM[J]. Computer Applications and Software, 2021, 38(12): 179-186.)
[20]
Bai S J, Kolter J Z, Koltun V. An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling[OL]. arXiv Preprint, arXiv: 1803.01271.
(Gao Wei, Ma Hui, Li Dazhou, et al. Research and Implementation of Chinese Text Abstract Technology Based on Double Encoder[J]. Computer Engineering and Design, 2021, 42(9): 2687-2695.)
[22]
Cao D, Huang Y J, Fu Y B. Text Sentiment Analysis Based on Parallel TCN Model and Attention Model[C]// Proceedings of the 2nd Symposium on Signal Processing Systems. ACM, 2020: 86-90.
(Zou Zhi, Wu Tiezhou, Zhang Xiaoxing, et al. Short-Term Load Forecast Based on Bayesian Optimized CNN-BiGRU Hybrid Neural Networks[J]. High Voltage Engineering, 2022, 48(10): 3935-3945.)
[24]
段明君. 基于生成对抗网络的中文文本生成[D]. 成都: 电子科技大学, 2021.
[24]
(Duan Mingjun. Research on Chinese Text Generation Based on Generative Adversarial Networks[D]. Chengdu: University of Electronic Science and Technology of China, 2021.)
[25]
Papineni K, Roukos S, Ward T, et al. BLEU: A Method for Automatic Evaluation of Machine Translation[C]// Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. New York: ACM, 2002: 311-318.
(He Tianwen, Wang Hong. Evaluating Perplexity of Chinese Sentences Based on Grammar & Semantics Analysis[J]. Application Research of Computers, 2017, 34(12): 3538-3542.)
[27]
Heafield K. KenLM: Faster and Smaller Language Model Queries[C]// Proceedings of the 6th Workshop on Statistical Machine Translation. ACM, 2011: 187-197.
[28]
Sutskever I, Vinyals O, Le Q V. Sequence to Sequence Learning with Neural Networks[OL]. arXiv Preprint, arXiv: 1409.3215.
[29]
Gehring J, Auli M, Grangier D, et al. Convolutional Sequence to Sequence Learning[C]// Proceedings of the 34th International Conference on Machine Learning. ACM, 2017: 1243-1252.
[30]
Lai S W, Xu L H, Liu K, et al. Recurrent Convolutional Neural Networks for Text Classification[C]// Proceedings of the 29th AAAI Conference on Artificial Intelligence. ACM, 2015: 2267-2273.
[31]
Srivastava R K, Greff K, Schmidhuber J. Highway Networks[OL]. arXiv Preprint, arXiv: 1505.00387.
[32]
Zhou P, Shi W, Tian J, et al. Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016: 207-212.
