[Objective] This paper proposes an information extraction model (SpERT-Aggcn) and constructs knowledge graphs for green cooperation patents based on this model. It helps us identify nested entities and improve the accuracy of relationship extraction for knowledge graphs. [Methods] First, we utilized the SpERT-Aggcn model to extract nested entities and relationships from patent abstracts. Then, we built an ontology using Protégé and mapped the triples with the constructed ontology. [Results] In relationship extraction, the SpERT-Aggcn model’s F1 score was 2.61% higher than the SpERT model. The SpERT-Aggcn model’s F1 score was 4.42% higher than the SpERT model for the long-distance relationship extraction tasks. The constructed knowledge graph for green cooperation patents contained 699,517 entities and 3,241,805 relationships. [Limitations] The F1 score of SpERT-Aggcn for extracting short-distance relationships was lower than the SpERT model, indicating a weaker capability of the proposed model in identifying short-distance relationships. [Conclusions] The proposed model could help us construct better knowledge graphs.
(Zero One Think Tank, Hengqin Digital Finance Research Institute. China Green Technology Innovation Index Report (2021)[EB/OL]. [2022-10-01]. https://www.01caijing.com/article/323551.htm.)
(Qi Shaozhou, Lin Shen, Cui Jingbo. Do Environmental Rights Trading Schemes Induce Green Innovation? Evidence from Listed Firms in China[J]. Economic Research Journal, 2018, 53(12): 129-143.)
(Tian Ling, Zhang Jinchuan, Zhang Jinhao, et al. Knowledge Graph Survey: Representation, Construction, Reasoning and Knowledge Hypergraph Theory[J]. Journal of Computer Applications, 2021, 41(8): 2161-2186.)
doi: 10.11772/j.issn.1001-9081.2021040662
[7]
Eberts M, Ulges A. Span-Based Joint Entity and Relation Extraction with Transformer Pre-Training[OL]. arXiv Preprint, arXiv: 1909.07755.
[8]
Guo Z J, Zhang Y, Lu W. Attention Guided Graph Convolutional Networks for Relation Extraction[OL]. arXiv Preprint, arXiv: 1906.07510.
[9]
Lafferty J, McCallum A, Peteira F C. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data[C]// Proceedings of the 18th International Conference on Machine Learning. 2001: 282-289.
[10]
Ma X Z, Hovy E. End-to-End Sequence Labeling via Bi-Directional LSTM-CNNS-CRF[OL]. arXiv Preprint, arXiv: 1603.01354.
[11]
Liu L Y, Shang J B, Ren X, et al. Empower Sequence Labeling with Task-Aware Neural Language Model[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence and 30th Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence. 2018: 5253-5260.
[12]
Fu J L, Huang X J, Liu P F. SpanNER: Named Entity Re-/Recognition as Span Prediction[C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 2021: 7183-7195.
[13]
Li F, Lin Z C, Zhang M S, et al. A Span-Based Model for Joint Overlapped and Discontinuous Named Entity Recognition[C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 2021: 4814-4828.
[14]
Zhong Z X, Chen D Q. A Frustratingly Easy Approach for Entity and Relation Extraction[C]// Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. 2021: 50-61.
[15]
Ye D M, Lin Y K, Li P, et al. Packed Levitated Marker for Entity and Relation Extraction[C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 2022: 4904-4917.
[16]
Zhang M S, Zhang Y, Fu G H. End-to-End Neural Relation Extraction with Global Optimization[C]// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017: 1730-1740.
[17]
Li X Y, Yin F, Sun Z J, et al. Entity-Relation Extraction as Multi-Turn Question Answering[OL]. arXiv Preprint, arXiv: 1905.05529.
[18]
Sun C Z, Gong Y Y, Wu Y B, et al. Joint Type Inference on Entities and Relations via Graph Convolutional Networks[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 1361-1370.
[19]
Zheng S C, Wang F, Bao H Y, et al. Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme[OL]. arXiv Preprint, arXiv: 1706.05075.
[20]
Bekoulis G, Deleu J, Demeester T, et al. Joint Entity Recognition and Relation Extraction as a Multi-Head Selection Problem[J]. Expert Systems with Applications, 2018, 114: 34-45.
doi: 10.1016/j.eswa.2018.07.032
[21]
Zhang Y H, Qi P, Manning C D. Graph Convolution over Pruned Dependency Trees Improves Relation Extraction[OL]. arXiv Preprint, arXiv: 1809.10185.
[22]
Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. ACM, 2017: 6000-6010.
[23]
Huang G, Liu Z, van der Maaten L, et al. Densely Connected Convolutional Networks[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2017: 2261-2269.
[24]
Singhal A. Introducing the Knowledge Graph: Things, Not Strings[EB/OL]. [2022-10-01]. https://blog.google/products/search/introducing-knowledge-graph-things-not/.
(Xu Zenglin, Sheng Yongpan, He Lirong, et al. Review on Knowledge Graph Techniques[J]. Journal of University of Electronic Science and Technology of China, 2016, 45(4): 589-606.)
(Lai Chaoan, Qian Jiao. A Method for Patent Mining Based on Knowledge Map and Its Application[J]. Science Research Management, 2017, 38(S1): 333-341.)
[28]
张少光. 面向创新的专利知识图谱构建与应用研究[D]. 天津: 河北工业大学, 2020.
[28]
(Zhang Shaoguang. Research on the Construction and Application of Innovation-Oriented Patent Knowledge Graph[D]. Tianjin: Hebei University of Technology, 2020.)
[29]
马国斌. 基于知识图谱的专利知识检索研究[D]. 哈尔滨: 哈尔滨工业大学, 2021.
[29]
(Ma Guobin. Research on Patent Knowledge Search Based on Knowledge Graph[D]. Harbin: Harbin Institute of Technology, 2021.)
[30]
吕向如. 中文专利知识图谱构建研究[D]. 北京: 北京信息科技大学, 2019.
[30]
(Lü Xiangru. Research on the Construction of Chinese Patent Knowledge Map[D]. Beijing: Beijing Information Science & Technology University, 2019.)
(Zhu Degang, Gong Lin, Tang Sheng, et al. Conceptual Design Method for Product Innovation Based on a Patent Knowledge Graph[J]. Computer Integrated Manufacturing Systems, 2022, 28(11): 3599-3614.)
[32]
林超. 基于自然语言处理的专利知识图谱构建研究[D]. 杭州: 杭州电子科技大学, 2021.
[32]
(Lin Chao. Research on the Construction of Patent Knowledge Graph Based on Natural Language Processing[D]. Hangzhou: Hangzhou Dianzi University, 2021.)
[33]
Devlin J, Chang M W, Lee K, et al. BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding[OL]. arXiv Preprint, arXiv: 1810.04805.
