Recognizing Chinese Medical Literature Entities Based on Multi-Task and Transfer Learning
Han Pu1,2(),Gu Liang1,Ye Dongyu1,Chen Wenqi1
1School of Management, Nanjing University of Posts & Telecommunications, Nanjing 210003, China 2Jiangsu Provincial Key Laboratory of Data Engineering and Knowledge Service, Nanjing 210023, China
[Objective] This paper uses transfer learning and multi-task learning to solve the problems of cold start and boundary in Chinese medical literature entity recognition, and further improve the recognition accuracy. [Methods] Firstly, we constructed a hybrid deep learning BERT-BiLSTM-IDCNN-CRF medical literature entity recognition model. Secondly, based on transfer learning, the medical semantic features were enriched through instance, model and feature transfer. Thirdly, we constructed a coarse-grained three-classification task through multi-task learning to assist the main task in utilizing the entity boundary information effectively. Finally, we introduced the self-attention mechanism and highway network to capture global information, optimize deep network training and establish the TLMT-BBIC-HS model. [Results] The model had an F1 value of 92.98% on the Chinese diabetes medical literature dataset, which is 15.99% and 16.44% higher than the benchmark models BERT-BiLSTM-CRF and BERT-IDCNN-CRF. [Limitations] The domain suitability of this model needs to be verified. [Conclusions] The TLMT-BBIC-HS model can transfer and share medical knowledge, which is more suitable for Chinese medical Literature entity recognition. It could effectively extract medical information and construct knowledge graphs and question answering systems.
韩普, 顾亮, 叶东宇, 陈文祺. 基于多任务和迁移学习的中文医学文献实体识别研究*[J]. 数据分析与知识发现, 2023, 7(9): 136-145.
Han Pu, Gu Liang, Ye Dongyu, Chen Wenqi. Recognizing Chinese Medical Literature Entities Based on Multi-Task and Transfer Learning. Data Analysis and Knowledge Discovery, 2023, 7(9): 136-145.
(Zhao Yang, Zhang Zhixiong, Liu Huan, et al. Classification of Chinese Medical Literature with BERT Model[J]. Data Analysis and Knowledge Discovery, 2020, 4(8): 41-49.)
(Li Yueyan, Wang Hao, Deng Sanhong, et al. Research on Semantic Relevance of Medical Text Oriented to Event Ontology[J]. Journal of the China Society for Scientific and Technical Information, 2022, 41(5): 497-511.)
[3]
Coden A, Savova G, Sominsky I, et al. Automatically Extracting Cancer Disease Characteristics from Pathology Reports into a Disease Knowledge Representation Model[J]. Journal of Biomedical Informatics, 2009, 42(5): 937-949.
doi: 10.1016/j.jbi.2008.12.005
pmid: 19135551
[4]
Jiang M, Chen Y K, Liu M, et al. A Study of Machine-Learning-Based Approaches to Extract Clinical Entities and Their Assertions from Discharge Summaries[J]. Journal of the American Medical Informatics Association, 2011, 18(5): 601-606.
doi: 10.1136/amiajnl-2011-000163
pmid: 21508414
[5]
Liu Z J, Yang M, Wang X L, et al. Entity Recognition from Clinical Texts via Recurrent Neural Network[J]. BMC Medical Informatics and Decision Making, 2017, 17(Suppl 2): Article No.67.
[6]
Gajendran S, Manjula D, Sugumaran V. Character Level and Word Level Embedding with Bidirectional LSTM—Dynamic Recurrent Neural Network for Biomedical Named Entity Recognition from Literature[J]. Journal of Biomedical Informatics, 2020, 112: Article No.103609.
[7]
Li X Y, Zhang H, Zhou X H. Chinese Clinical Named Entity Recognition with Variant Neural Structures Based on BERT Methods[J]. Journal of Biomedical Informatics, 2020, 107: Article No.103422.
(Lü Jianghai, Du Junping, Zhou Nan, et al. Entity Name Recognition Method Based on Dilated Convolutional Iterative and Attention Mechanism[J]. Computer Engineering, 2021, 47(1): 58-65.)
doi: 10.19678/j.issn.1000-3428.0055986
[9]
Giorgi J M, Bader G D. Transfer Learning for Biomedical Named Entity Recognition with Neural Networks[J]. Bioinformatics, 2018, 34(23): 4087-4094.
doi: 10.1093/bioinformatics/bty449
pmid: 29868832
[10]
Smetanin S, Komarov M. Deep Transfer Learning Baselines for Sentiment Analysis in Russian[J]. Information Processing & Management, 2021, 58(3): Article No.102484.
[11]
Fu W L, Xue B, Gao X Y, et al. Transductive Transfer Learning Based Genetic Programming for Balanced and Unbalanced Document Classification Using Different Types of Features[J]. Applied Soft Computing, 2021, 103: Article No.107172.
[12]
Mignone P, Pio G, D’Elia D, et al. Exploiting Transfer Learning for the Reconstruction of the Human Gene Regulatory Network[J]. Bioinformatics, 2020, 36(5): 1553-1561.
doi: 10.1093/bioinformatics/btz781
pmid: 31608946
(Xiong Xin, Wang Hao, Deng Sanhong. A Study on Term Extraction Model with Transfer Learning for Knowledge Graph of Local Chronicles[J]. Information Studies: Theory & Application, 2021, 44(4): 176-184.)
(Han Pu, Zhang Zhanpeng, Zhang Wei. Chinese Disease Name Normalization Based on Multi-Task Learning and Polymorphic Semantic Features[J]. Journal of the China Society for Scientific and Technical Information, 2021, 40(11): 1234-1244.)
[15]
Crichton G, Pyysalo S, Chiu B, et al. A Neural Network Multi-Task Learning Approach to Biomedical Named Entity Recognition[J]. BMC Bioinformatics, 2017, 18(1): 1-14.
[16]
Wu C C, Luo G, Guo C, et al. An Attention-Based Multi-Task Model for Named Entity Recognition and Intent Analysis of Chinese Online Medical Questions[J]. Journal of Biomedical Informatics, 2020, 108: Article No.103511.
[17]
Aguilar G, Maharjan S, López-Monroy A P, et al. A Multi-Task Approach for Named Entity Recognition in Social Media Data[OL]. arXiv Preprint, arXiv: 1906.04135.
[18]
Wang D S, Fan H J, Liu J F. Learning with Joint Cross-Document Information via Multi-Task Learning for Named Entity Recognition[J]. Information Sciences, 2021, 579: 454-467.
doi: 10.1016/j.ins.2021.08.015
[19]
Srivastava R K, Greff K, Schmidhuber J. Highway Networks[OL]. arXiv Preprint, arXiv: 1505.00387.
[20]
Zuo M, Zhang Y. Dataset-Aware Multi-Task Learning Approaches for Biomedical Named Entity Recognition[J]. Bioinformatics, 2020, 36(15): 4331-4338.
doi: 10.1093/bioinformatics/btaa515
pmid: 32415963
[21]
Liu L Y, Shang J B, Ren X A, et al. Empower Sequence Labeling with Task-Aware Neural Language Model[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 2018, 32(1): 5253-5260.
[22]
Narayanan S, Achan P, Rangan P V, et al. Unified Concept and Assertion Detection Using Contextual Multi-Task Learning in a Clinical Decision Support System[J]. Journal of Biomedical Informatics, 2021, 122: Article No.103898.
(Wang Dongbo, Liu Chang, Zhu Zihe, et al. Construction and Application of Pre-Trained Models of Siku Quanshu in Orientation to Digital Humanities[J]. Library Tribune, 2022, 42(6): 31-43.)
[24]
Aliyun. A Labeled Chinese Dataset for Diabetes[EB/OL]. [2022-06-28]. https://tianchi.aliyun.com/competition/entrance/231687/information.
[25]
Aya Mohamed Abdelaty Elkased. 面向生物医学文献的基于BioBERT的药品相互作用抽取增强模型[D]. 哈尔滨: 哈尔滨工业大学, 2021.
[25]
(Aya Mohamed Abdelaty Elkased. Enhanced Drug-Drug Interaction Extraction Model from Biomedical Text Using BioBERT[D]. Harbin: Harbin Institute of Technology, 2021.)
He Chunhui, Wang Mengxian, He Xiaobo. Named Entity Recognition in the Field of Diabetes Based on Double-layer Bi-LSTM-CRF Model[J]. Journal of Shaoyang University (Natural Science Edition), 2020, 17(1): 21-26.)
[27]
Shang F J, Ran C F. An Entity Recognition Model Based on Deep Learning Fusion of Text Feature[J]. Information Processing & Management, 2022, 59(2): Article No.102841.
[28]
Deng J F, Cheng L L, Wang Z W. Self-Attention-Based BiGRU and Capsule Network for Named Entity Recognition[OL]. arXiv Preprint, arXiv: 2002.00735.
[29]
Wang Y, Sun Y N, Ma Z C, et al. Named Entity Recognition in Chinese Medical Literature Using Pretraining Models[J]. Scientific Programming, 2020, 2020: Article No.8812754.
