1Beijing Key Laboratory of Internet Culture and Digital Dissemination Research, Beijing Information Science & Technology University, Beijing 100101, China 2School of Information & Communication Engineering, Beijing Information Science & Technology University, Beijing 100101, China
[Objective] This paper proposes a rewriting model for Chinese patent abstracts based on reinforcement learning (RLCPAR), aiming to address the issues of sentence redundancy and low accuracy in rewriting multi-sentence abstracts. [Methods] First, we used the RLCPAR to extract key sentences from patent descriptions with the help of patent term dictionary and reinforcement learning. Then, we generated the candidate abstracts using the Transformer deep neural network. Finally, we merged the candidate abstracts with the original patent abstracts to obtain the rewritten abstracts after semantic de-duplication and sorting. [Results] The proposed model effectively finished the end-to-end rewriting of patent abstracts. The scores of RLCPAR were 56.95%, 37.21% and 51.24% with the ROUGE-1, ROUGE-2 and ROUGE-L criteria. [Limitations] The experimental data, which were mainly on Chinese medicine materials, needs to be expanded to other fields. [Conclusions] The PLCPAR model is much better than other sequence generation methods and improves the rewriting quality of Chinese patent abstracts.
张乐, 冷基栋, 吕学强, 崔卓, 王磊, 游新冬. RLCPAR：一种基于强化学习的中文专利摘要改写模型*[J]. 数据分析与知识发现, 2021, 5(7): 59-69.
Zhang Le, Leng Jidong, Lv Xueqiang, Cui Zhuo, Wang Lei, You Xindong. RLCPAR: A Rewriting Model for Chinese Patent Abstracts Based on Reinforcement Learning. Data Analysis and Knowledge Discovery, 2021, 5(7): 59-69.
(Zhang Jinzhu, Zhu Lipeng, Liu Jingjie. Unsupervised Cross-Language Model for Patent Recommendation Based on Representation[J]. Data Analysis and Knowledge Discovery, 2020, 4(10):93-103.)
(Shi Chaoyang. Discussion on the “Beneficial Effect” in the Rewriting of Chinese Patent Abstract[J]. Patent Literature Research, 2007(2):9-13.)
[4]
Berg-Kirkpatrick T, Gillick D, Klein D. Jointly Learning to Extract and Compress[C]// Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011:481-490.
[5]
Banko M, Mittal V O, Witbrock M J. Headline Generation Based on Statistical Translation[C]// Proceedings of the 38th Annual Meeting on Association for Computational Linguistics. 2000:318-325.
[6]
Radev D R, Jing H, Sty M, et al. Centroid-based Summarization of Multiple Documents[J]. Information Processing & Management, 2004, 40(6):919-938.
doi: 10.1016/j.ipm.2003.10.006
[7]
Knight K, Marcu D. Statistics-Based Summarization -Step One: Sentence Compression[C]// Proceedings of the 17th National Conference on Artificial Intelligence and 12th Conference on Innovative Applications of Artificial Intelligence. Austin, Texas: AAAI Press, 2000:703-710.
[8]
Gillick D, Favre B. A Scalable Global Model for Summarization[C]// Proceedings of the Workshop on Integer Linear Programming for Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2009: 10-18.
[9]
Marcu D. The Theory and Practice of Discourse Parsing and Summarization by Daniel Marcu[J]. Computational Linguistics, 2000, 28(1):81-83.
doi: 10.1162/coli.2000.28.1.81
[10]
Pera M S, Ng Y K. A Naive Bayes Classifier for Web Document Summaries Created by Using Word Similarity and Significant Factors[J]. International Journal on Artificial Intelligence Tools, 2010, 19(4):465-486.
doi: 10.1142/S0218213010000285
(Zhang Han, Zhao Yuhong. Constructing Semantic Graph Based on Summary Extracting Model for Multiple for Multiple Medical Documents[J]. Library and Information Service, 2017, 61(8):112-119.)
[12]
Yan S, Wan X J. SRRank: Leveraging Semantic Roles for Extractive Multi-Document Summarization[J]. IEEE-ACM Transactions on Audio Speech and Language Processing, 2014, 22(12):2048-2058.
[13]
Cheng J P, Lapata M. Neural Summarization by Extracting Sentences and Words[C]// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2016: 484-494.
(Jia Xiaoting, Wang Mingyang, Cao Yu. Automatic Abstracting of Chinese Document with Doc2Vec and Improved Clustering Algorithm[J]. Data Analysis and Knowledge Discovery, 2018, 2(2):86-95.)
(Zhang Ying, Wang Zhongqing, Wang Hongling. Single Document Extractive Summarization with Satellite and Nuclear Relations[J]. Journal of Chinese Information Processing, 2019, 33(8):67-76.)
[16]
Liu F, Flanigan J, Thomson S, et al. Toward Abstractive Summarization Using Semantic Representations[C]// Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg: Association for Computational Linguistics, 2015: 1077-1086.
[17]
Bing L D, Li P J, Liao Y, et al. Abstractive Multi-Document Summarization via Phrase Selection and Merging[J]. Computational Linguistics, 2015, 31(4):505-530.
doi: 10.1162/089120105775299140
[18]
Nallapati R, Zhou B W, Dos Santos C N, et al. Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond[C]// Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning. Stroudsburg: Association for Computational Linguistics, 2016: 280-290.
[19]
See A, Liu P J, Manning C D. Get to the Point: Summarization with Pointer-generator Networks[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2017: 1073-1083.
(Shen Si, Hu Haotian, Ye Wenhao, et al. Research on Abstract Structure Function Automatic Recognition Based on Full Character Semantics[J]. Journal of the China Society for Scientific and Technical Information, 2019, 38(1):79-88.)
[21]
Wang K, Quan X J, Wang R. BiSET: Bi-directional Selective Encoding with Template for Abstractive Summarization[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Stroudsburg: Association for Computational Linguistics, 2019: 2153-2162.
[22]
Song K T, Tan X, Qin T, et al. Mass: Masked Sequence to Sequence Pre-training for Language Generation[C]// Proceedings of the 36th International Conference on Machine Learning. California:International Conference on Machine Learning, 2019: 5926-5936.
[23]
Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook: Curran Associates, 2017: 6000-6010.
[24]
Le Q V, Mikolov T. Distributed Representations of Sentences and Documents[C]// Proceedings of the 31st International Conference on Machine Learning. New York: ACM, 2014: 1188-1196.
[25]
Vinyals O, Fortunato M, Jaitly N. Pointer Networks[C]// Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge, MA: MIT Press, 2015: 2692-2700.
[26]
Mnih V, Badia A P, Mirza M, et al. Asynchronous Methods for Deep Reinforcement Learning[C]// Proceedings of the 33rd International Conference on Machine Learning. USA: International Machine Learning Society, 2016: 2850-2869.
[27]
Paulus R, Xiong C, Socher R. A Deep Reinforced Model for Abstractive Summarization[C]// Proceedings of International Conference on Learning Representations. 2018: 428-441.
[28]
Lin C Y. ROUGE: A Package for Automatic Evaluation of Summaries[C]// Proceedings of the Workshop on Text Summarization Branches Out (WAS 2004). Stroudsburg: Association for Computational Linguistics, 2004: 74-81.
[29]
Chen Y C, Bansal M. Fast Abstractive Summarization with Rein-force-selected Sentence Rewriting[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: Association for Computational Linguistics, 2018: 675-686.
