Please wait a minute...
Data Analysis and Knowledge Discovery  2024, Vol. 8 Issue (1): 146-156    DOI: 10.11925/infotech.2096-3467.2022.1142
Current Issue | Archive | Adv Search |
Constructing Patent Knowledge Graph with SpERT-Aggcn Model
He Yu,Zhang Xiaodong(),Zheng Xin
School of Economics and Management, University of Science and Technology Bejing, Beijing 100083, China
Download: PDF (1502 KB)   HTML ( 18 ) 
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper proposes an information extraction model (SpERT-Aggcn) and constructs knowledge graphs for green cooperation patents based on this model. It helps us identify nested entities and improve the accuracy of relationship extraction for knowledge graphs. [Methods] First, we utilized the SpERT-Aggcn model to extract nested entities and relationships from patent abstracts. Then, we built an ontology using Protégé and mapped the triples with the constructed ontology. [Results] In relationship extraction, the SpERT-Aggcn model’s F1 score was 2.61% higher than the SpERT model. The SpERT-Aggcn model’s F1 score was 4.42% higher than the SpERT model for the long-distance relationship extraction tasks. The constructed knowledge graph for green cooperation patents contained 699,517 entities and 3,241,805 relationships. [Limitations] The F1 score of SpERT-Aggcn for extracting short-distance relationships was lower than the SpERT model, indicating a weaker capability of the proposed model in identifying short-distance relationships. [Conclusions] The proposed model could help us construct better knowledge graphs.
Key words： Green Cooperation Patent      Knowledge Graph      Graph Convolution Network      Information Extraction     
Received: 31 October 2022      Published: 28 April 2023
ZTFLH:  TP393  
  G250  
Fund:National Natural Science Foundation of China(71871018)
Corresponding Authors: Zhang Xiaodong，ORCID：0000-0002-8203-9763，E-mail：xdzhang@manage.ustb.edu.cn。   
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Yu He
Xiaodong Zhang
Xin Zheng

Cite this article:

He Yu, Zhang Xiaodong, Zheng Xin. Constructing Patent Knowledge Graph with SpERT-Aggcn Model. Data Analysis and Knowledge Discovery, 2024, 8(1): 146-156.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2022.1142     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2024/V8/I1/146

Difference Between Sequence Annotation Model and Span Based Model
Construction Process of Green Cooperation Patent Knowledge Graph
Model Structure of SpERT-Aggcn
Entity Recognition Module of SpERT-Aggcn
Aggcn Module
Relation Extraction Module
模型 实体识别 关系抽取
Precision（%） Recall（%） F1（%） Precision（%） Recall（%） F1（%）
SpERT 79.25 83.39 81.27 48.82 68.21 56.91
SpERT-Aggcn 82.01 81.01 81.51 52.92 68.00 59.52
Result of Test Dataset
模型 专利-优点：（间隔长度为45字） 专利-设备：（间隔长度为42字） 空间关系：（间隔长度为12字） 设备-原料：（间隔长度为5字）
SpERT 64.46 68.49 52.87 41.07
SpERT-Aggcn 68.88 70.17 54.72 40.61
F1 Value of Relation Classification
ID 问题类型 实例
实例1 漏标纠正 [本实用新型][结构简单]，[对污泥处理效果好]，[充分利用能源]
实例2 语义错误 [本发明]公开了一种以[[转炉][钢渣]]为原料制备[[钙铁]双氧载体]的方法，属于[固体废弃物]利用技术领域
实例3 实体漏检 [本发明]涉及一种[[餐饮[垃圾]]自动粉碎压榨一体化设备]
实例4 关系漏检 [本发明]专利-优点能够[提高高压环路互锁系统可靠性和稳定性]专利-优点
Instance of Bad Case
Ontology of Green Cooperation Patent Knowledge Graph
Knowledge Graph Visualization Based on Neo4j
[1] 程文婷. 绿色专利[J]. 电子知识产权, 2011(Z1): 42.
[1] (Cheng Wenting. Green Patent[J]. Electronics Intellectual Property, 2011(Z1): 42.)
[2] 零壹智库, 横琴数链数字金融研究院. 中国绿色技术创新指数报告(2021)[EB/OL]. [2022-10-01]. https://www.01caijing.com/article/323551.htm.
[2] (Zero One Think Tank, Hengqin Digital Finance Research Institute. China Green Technology Innovation Index Report (2021)[EB/OL]. [2022-10-01]. https://www.01caijing.com/article/323551.htm.)
[3] 齐绍洲, 林屾, 崔静波. 环境权益交易市场能否诱发绿色创新?——基于我国上市公司绿色专利数据的证据[J]. 经济研究, 2018, 53(12): 129-143.
[3] (Qi Shaozhou, Lin Shen, Cui Jingbo. Do Environmental Rights Trading Schemes Induce Green Innovation? Evidence from Listed Firms in China[J]. Economic Research Journal, 2018, 53(12): 129-143.)
[4] 王班班, 赵程. 中国的绿色技术创新——专利统计和影响因素[J]. 工业技术经济, 2019, 38(7): 53-66.
doi: 10.3969/j.issn.1004-910X.2019.07.007
[4] (Wang Banban, Zhao Cheng. China’s Green Technological Innovation—Patent Statistics and Influencing Factors[J]. Journal of Industrial Technological Economics, 2019, 38(7): 53-66.)
doi: 10.3969/j.issn.1004-910X.2019.07.007
[5] 安晓慧, 许轶. 绿色专利技术创新趋势[J]. 世界科技研究与发展, 2020, 42(3): 358.
[5] (An Xiaohui, Xu Yi. Innovation Trend of Green Patent Technology[J]. World Sci-Tech R&D, 2020, 42(3): 358.)
[6] 田玲, 张谨川, 张晋豪, 等. 知识图谱综述——表示、构建、推理与知识超图理论[J]. 计算机应用, 2021, 41(8): 2161-2186.
doi: 10.11772/j.issn.1001-9081.2021040662
[6] (Tian Ling, Zhang Jinchuan, Zhang Jinhao, et al. Knowledge Graph Survey: Representation, Construction, Reasoning and Knowledge Hypergraph Theory[J]. Journal of Computer Applications, 2021, 41(8): 2161-2186.)
doi: 10.11772/j.issn.1001-9081.2021040662
[7] Eberts M, Ulges A. Span-Based Joint Entity and Relation Extraction with Transformer Pre-Training[OL]. arXiv Preprint, arXiv: 1909.07755.
[8] Guo Z J, Zhang Y, Lu W. Attention Guided Graph Convolutional Networks for Relation Extraction[OL]. arXiv Preprint, arXiv: 1906.07510.
[9] Lafferty J, McCallum A, Peteira F C. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data[C]// Proceedings of the 18th International Conference on Machine Learning. 2001: 282-289.
[10] Ma X Z, Hovy E. End-to-End Sequence Labeling via Bi-Directional LSTM-CNNS-CRF[OL]. arXiv Preprint, arXiv: 1603.01354.
[11] Liu L Y, Shang J B, Ren X, et al. Empower Sequence Labeling with Task-Aware Neural Language Model[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence and 30th Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence. 2018: 5253-5260.
[12] Fu J L, Huang X J, Liu P F. SpanNER: Named Entity Re-/Recognition as Span Prediction[C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 2021: 7183-7195.
[13] Li F, Lin Z C, Zhang M S, et al. A Span-Based Model for Joint Overlapped and Discontinuous Named Entity Recognition[C]// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 2021: 4814-4828.
[14] Zhong Z X, Chen D Q. A Frustratingly Easy Approach for Entity and Relation Extraction[C]// Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. 2021: 50-61.
[15] Ye D M, Lin Y K, Li P, et al. Packed Levitated Marker for Entity and Relation Extraction[C]// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. 2022: 4904-4917.
[16] Zhang M S, Zhang Y, Fu G H. End-to-End Neural Relation Extraction with Global Optimization[C]// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017: 1730-1740.
[17] Li X Y, Yin F, Sun Z J, et al. Entity-Relation Extraction as Multi-Turn Question Answering[OL]. arXiv Preprint, arXiv: 1905.05529.
[18] Sun C Z, Gong Y Y, Wu Y B, et al. Joint Type Inference on Entities and Relations via Graph Convolutional Networks[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 2019: 1361-1370.
[19] Zheng S C, Wang F, Bao H Y, et al. Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme[OL]. arXiv Preprint, arXiv: 1706.05075.
[20] Bekoulis G, Deleu J, Demeester T, et al. Joint Entity Recognition and Relation Extraction as a Multi-Head Selection Problem[J]. Expert Systems with Applications, 2018, 114: 34-45.
doi: 10.1016/j.eswa.2018.07.032
[21] Zhang Y H, Qi P, Manning C D. Graph Convolution over Pruned Dependency Trees Improves Relation Extraction[OL]. arXiv Preprint, arXiv: 1809.10185.
[22] Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. ACM, 2017: 6000-6010.
[23] Huang G, Liu Z, van der Maaten L, et al. Densely Connected Convolutional Networks[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2017: 2261-2269.
[24] Singhal A. Introducing the Knowledge Graph: Things, Not Strings[EB/OL]. [2022-10-01]. https://blog.google/products/search/introducing-knowledge-graph-things-not/.
[25] 徐增林, 盛泳潘, 贺丽荣, 等. 知识图谱技术综述[J]. 电子科技大学学报, 2016, 45(4): 589-606.
[25] (Xu Zenglin, Sheng Yongpan, He Lirong, et al. Review on Knowledge Graph Techniques[J]. Journal of University of Electronic Science and Technology of China, 2016, 45(4): 589-606.)
[26] 邵泽宇, 孟天宇. 基于知识图谱的区块链专利数据挖掘[J]. 技术与创新管理, 2020, 41(6): 588-595.
[26] (Shao Zeyu, Meng Tianyu. Block Chain Patented Data Mining Based on Mapping Knowledge[J]. Technology and Innovation Management, 2020, 41(6): 588-595.)
[27] 赖朝安, 钱娇. 基于知识图谱的专利挖掘方法及其应用[J]. 科研管理, 2017, 38(S1): 333-341.
[27] (Lai Chaoan, Qian Jiao. A Method for Patent Mining Based on Knowledge Map and Its Application[J]. Science Research Management, 2017, 38(S1): 333-341.)
[28] 张少光. 面向创新的专利知识图谱构建与应用研究[D]. 天津: 河北工业大学, 2020.
[28] (Zhang Shaoguang. Research on the Construction and Application of Innovation-Oriented Patent Knowledge Graph[D]. Tianjin: Hebei University of Technology, 2020.)
[29] 马国斌. 基于知识图谱的专利知识检索研究[D]. 哈尔滨: 哈尔滨工业大学, 2021.
[29] (Ma Guobin. Research on Patent Knowledge Search Based on Knowledge Graph[D]. Harbin: Harbin Institute of Technology, 2021.)
[30] 吕向如. 中文专利知识图谱构建研究[D]. 北京: 北京信息科技大学, 2019.
[30] (Lü Xiangru. Research on the Construction of Chinese Patent Knowledge Map[D]. Beijing: Beijing Information Science & Technology University, 2019.)
[31] 祝德刚, 宫琳, 唐圣, 等. 基于专利知识图谱的产品创新概念设计方法[J]. 计算机集成制造系统, 2022, 28(11): 3599-3614.
[31] (Zhu Degang, Gong Lin, Tang Sheng, et al. Conceptual Design Method for Product Innovation Based on a Patent Knowledge Graph[J]. Computer Integrated Manufacturing Systems, 2022, 28(11): 3599-3614.)
[32] 林超. 基于自然语言处理的专利知识图谱构建研究[D]. 杭州: 杭州电子科技大学, 2021.
[32] (Lin Chao. Research on the Construction of Patent Knowledge Graph Based on Natural Language Processing[D]. Hangzhou: Hangzhou Dianzi University, 2021.)
[33] Devlin J, Chang M W, Lee K, et al. BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding[OL]. arXiv Preprint, arXiv: 1810.04805.
[34] 刘峤, 李杨, 段宏, 等. 知识图谱构建技术综述[J]. 计算机研究与发展, 2016, 53(3): 582-600.
[34] (Liu Qiao, Li Yang, Duan Hong, et al. Knowledge Graph Construction Techniques[J]. Journal of Computer Research and Development, 2016, 53(3): 582-600.)
[1] Zhai Dongsheng, Lou Ying, Kan Huimin, He Xijun, Liang Guoqiang, Ma Zifei. Constructing TCM Knowledge Graph with Multi-Source Heterogeneous Data[J]. 数据分析与知识发现, 2023, 7(9): 146-158.
[2] Bao Tong, Zhang Chengzhi. Extracting Chinese Information with ChatGPT：An Empirical Study by Three Typical Tasks[J]. 数据分析与知识发现, 2023, 7(9): 1-11.
[3] Zhang Yingyi, Zhang Chengzhi, Zhou Yi, Chen Bikun. ChatGPT-Based Scientific Paper Entity Recognition: Performance Measurement and Availability Research[J]. 数据分析与知识发现, 2023, 7(9): 12-24.
[4] Zhang Zhijian, Ni Zhenni, Liu Zhenghao, Xia Sudi. Predicting Dynamic Relationship for Financial Knowledge Graph[J]. 数据分析与知识发现, 2023, 7(9): 39-50.
[5] Pu Xianghe, Wang Hongbin, Xian Yantuan. Few-Shot Knowledge Graph Completion Combined with Type-Aware Attention[J]. 数据分析与知识发现, 2023, 7(9): 51-63.
[6] Wang Xiaofeng, Sun Yujie, Wang Huazhen, Zhang Hengzhang. Construction and Verification of Type-Controllable Question Generation Model Based on Deep Learning and Knowledge Graphs[J]. 数据分析与知识发现, 2023, 7(6): 26-37.
[7] Li Kaijun, Niu Zhendong, Shi Kaize, Qiu Ping. Paper Recommendation Based on Academic Knowledge Graph and Subject Feature Embedding[J]. 数据分析与知识发现, 2023, 7(5): 48-59.
[8] Wang Yinqiu, Yu Wei, Chen Junpeng. Automatic Question-Answering in Chinese Medical Q & A Community with Knowledge Graph[J]. 数据分析与知识发现, 2023, 7(3): 97-109.
[9] Du Yue, Chang Zhijun, Dong Mei, Qian Li, Wang Ying. Constructing Large-scale Knowledge Graph for Massive Sci-Tech Literature[J]. 数据分析与知识发现, 2023, 7(2): 141-150.
[10] Zhang Zhengang, Yu Chuanming. Knowledge Graph Completion Model Based on Entity and Relation Fusion[J]. 数据分析与知识发现, 2023, 7(2): 15-25.
[11] Wei Jianxiang, Lu Qian, Han Pu, Huang Weidong. Event Detection Model Based on Semantic Information Fusion[J]. 数据分析与知识发现, 2023, 7(12): 64-74.
[12] Lyu Xueqiang, Du Yifan, Zhang Le, Pan Huiping, Tian Chi. GKTR Retrieval Model for Engineering Consulting Reports with Graph Convolution Topological and Keyword Features[J]. 数据分析与知识发现, 2023, 7(12): 155-163.
[13] Hua Bin, Wei Menghan. Research and Practice of Reasoning-Assisted Decision-Making Methods for Injury Crimes[J]. 数据分析与知识发现, 2023, 7(12): 142-154.
[14] Liu Shuai, Fu Lifang. Identifying Fake News with External Knowledge and User Interaction Features[J]. 数据分析与知识发现, 2023, 7(11): 79-87.
[15] Peng Cheng, Zhang Chunxia, Zhang Xin, Guo Jingtao, Niu Zhendong. Reasoning Model for Temporal Knowledge Graph Based on Entity Multiple Unit Coding[J]. 数据分析与知识发现, 2023, 7(1): 138-149.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626，82624938   E-mail:jishu@mail.las.ac.cn