Please wait a minute...
Data Analysis and Knowledge Discovery
Current Issue | Archive | Adv Search |
Chinese Electronic Medical Record Named Entity Recognition Based on RoBERTa-wwm Dynamic Fusion Model
Zhang Yunqiu,Wang Yang,Li Bocheng
(School of Public Health, Jilin University, Changchun, 130021, China)
Download:
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective]An entity recognition model based on RoBERTa-wwm dynamic fusion is proposed to improve the entity recognition effect of Chinese electronic medical records.

[Methods]After dynamicly fusing the semantic representations generated by each Transformer layer of the pre-trained language model RoBERTa-wwm,it’s inputed to the bi-directional long short-term memory network and the conditional random field module to complete the entity recognition in the electronic medical record.

[Results]In the "2017 National Knowledge Graph and Semantic Computing Conference (CCKS 2017)" data set and self-annotated electronic medical record data set,we get F1 values of 94.08% and 90.08% respectively. Compared to the RoBERTa-wwm-BiLSTM-CRF model, it improves

0.23% and 0.39%.

[Limitations]The RoBERTa-wwm used in this article completes the pre-training process based on non-medical corpus.

[Conclusions]The dynamic fusion of the semantic layer can make better use of the different information of each coding layer, and improve the effect of downstream entity recognition tasks.


Key words electronic medical record      named entity recognition      RoBERTa-wwm      dynamic fusion      
Published: 07 January 2022
ZTFLH:  TP391  

Cite this article:

Zhang Yunqiu, Wang Yang, Li Bocheng. Chinese Electronic Medical Record Named Entity Recognition Based on RoBERTa-wwm Dynamic Fusion Model . Data Analysis and Knowledge Discovery, 0, (): 1-.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2021.0951     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y0/V/I/1

[1] Xu Chenfei, Ye Haiying, Bao Ping. Automatic Recognition of Produce Entities from Local Chronicles with Deep Learning[J]. 数据分析与知识发现, 2020, 4(8): 86-97.
[2] Gao Yuan,Shi Yuanlei,Zhang Lei,Cao Tianyi,Feng Jun. Reconstructing Tour Routes Based on Travel Notes[J]. 数据分析与知识发现, 2020, 4(2/3): 165-172.
[3] Ma Jianxia,Yuan Hui,Jiang Xiang. Extracting Name Entities from Ecological Restoration Literature with Bi-LSTM+CRF[J]. 数据分析与知识发现, 2020, 4(2/3): 78-88.
[4] Liu Jingru,Song Yang,Jia Rui,Zhang Yipeng,Luo Yong,Ma Jingdong. A BiLSTM-CRF Model for Protected Health Information in Chinese[J]. 数据分析与知识发现, 2020, 4(10): 124-133.
[5] Jiahui Hu,An Fang,Wanqing Zhao,Chenliu Yang,Huiling Ren. Annotating Chinese E-Medical Record for Knowledge Discovery[J]. 数据分析与知识发现, 2019, 3(7): 123-132.
[6] Han Huang,Hongyu Wang,Xiaoguang Wang. Automatic Recognizing Legal Terminologies with Active Learning and Conditional Random Field Model[J]. 数据分析与知识发现, 2019, 3(6): 66-74.
[7] Kan Liu,Lu Chen. Deep Neural Network Learning for Medical Triage[J]. 数据分析与知识发现, 2019, 3(6): 99-108.
[8] Meishan Chen,Chenxi Xia. Identifying Entities of Online Questions from Cancer Patients Based on Transfer Learning[J]. 数据分析与知识发现, 2019, 3(12): 61-69.
[9] Li Yu,Li Qian,Changlei Fu,Huaming Zhao. Extracting Fine-grained Knowledge Units from Texts with Deep Learning[J]. 数据分析与知识发现, 2019, 3(1): 38-45.
[10] Tang Huihui,Wang Hao,Zhang Zixuan,Wang Xueying. Extracting Names of Historical Events Based on Chinese Character Tags[J]. 数据分析与知识发现, 2018, 2(7): 89-100.
[11] Fan Xinyue,Cui Lei. Using Text Mining to Discover Drug Side Effects: Case Study of PubMed[J]. 数据分析与知识发现, 2018, 2(3): 79-86.
[12] Mu Dongmei,Wang Ping,Zhao Danning. Reducing Data Dimension of Electronic Medical Records: An Empirical Study[J]. 数据分析与知识发现, 2018, 2(1): 88-98.
[13] Mu Dongmei,Ren Ke. Discovering Knowledge from Electronic Medical Records with Three Data Mining Algorithms[J]. 现代图书情报技术, 2016, 32(6): 102-109.
[14] Li Guolei, Chen Xianlai, Xia Dong, Yang Rong. Latent Semantic Analysis of Electronic Medical Record Text for Clinical Decision Making[J]. 数据分析与知识发现, 2016, 32(3): 50-57.
[15] Sui Mingshuang,Cui Lei. Extracting Chemical and Disease Named Entities with Multiple-Feature CRF Model[J]. 现代图书情报技术, 2016, 32(10): 91-97.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn