Improvement of Data Augment Algorithm for Named Entity Recognition with Small Samples

doi:10.11925/infotech.2096-3467.2022.0261

Data Analysis and Knowledge Discovery

2022, Vol. 6

Issue (10): 128-141 DOI: 10.11925/infotech.2096-3467.2022.0261

Current Issue | Archive | Adv Search

Improvement of Data Augment Algorithm for Named Entity Recognition with Small Samples

Liu Xingli^1,²(

),Fan Junjie²,Ma Haiqun¹

¹Research Center of Information Resources Management, Heilongjiang University, Harbin 150080, China
²School of Computer and Information Engineering, Heilongjiang University of Science and Technology, Harbin 150020, China

Download: PDF (1778 KB) HTML ( 14 )
Export: BibTeX | EndNote (RIS)

Abstract

[Objective] This paper proposes a strategy to improve data augment algorithm for named entities recognition with small samples. [Methods] Taking the task of domain named entity recognition as an example, a multi-dimensional improvement strategy based on easy data augment (EDA) algorithm is proposed: the entity replacement of mixed multiple domain dictionaries, the replacement of part of speech in domain semantic classification dictionaries, the random deletion based on semantic protection mechanism, the random insertion strategy of part of speech protection and the improved combination strategy of the four methods mentioned above, and the improved combination strategy of the four methods are respectively trained with named entity recognition(NER) model. [Results] The domain NER experimental results with small samples show that on the one hand, the efficiency was improved through a single strategy EDA: the F value is increased by 3.2, 4.6, 4.5 and 2.5 percentage points respectively. In contrast, the F value showed poor performance when applying two or more hybrid strategies. In the expansion experiment of the People’s Daily and Weibo datasets with small samples, the improvement effect was significant. The F value of the Entity Replacement Strategy Based on Multi-Domain Dictionary Mixing improvement strategy on the two datasets increased by 6.7 percentage points at the most. [Limitations] In the multiple strategy combination experiment, the regulation of the parameters α、N becomes more difficult, and the NER improvement of the combined strategy is affected. [Conclusions] The improvement strategy of EDA algorithm suggested in this paper effectively improves the results of named entity recognition model with small samples.

Method

Key words： Data Augment EDA Small Sample Name Entity Recognition

Received: 27 March 2022 Published: 09 November 2022

ZTFLH:

TP393 G250

Fund:National Social Science Fund of China(21&ZD336);National Social Science Fund of China(20ATQ004)

Corresponding Authors: Liu Xingli,ORCID：0000-0001-6126-9837 E-mail: liuxingli@usth.edu.cn

	Service

	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Xingli Liu
	Junjie Fan
	Haiqun Ma

Cite this article:

Liu Xingli, Fan Junjie, Ma Haiqun. Improvement of Data Augment Algorithm for Named Entity Recognition with Small Samples. Data Analysis and Knowledge Discovery, 2022, 6(10): 128-141.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2022.0261 OR https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2022/V6/I10/128

The Framework of Small Sample NER Based on EDA Improved Algorithm

The Improved MDDM Entity Replacement Strategy Design

The Example of Improved Strategy

The Dependency Tree Example of the Corpus

The Advantage and Disadvantage of the Improved Algorithm Comparison

The Framework of RoBERTa-WWM-BiLSTM-CRF

The Sample and Annotation Criterion of Domain Entities

The Experimental Configuration

The Experimental Environment

The Experiment of MDDM Entity Replacement Strategy Under Different Parameters

DA Techniques	$P w - a v g$ （%）	$R w - a v g (%)$	$F w - a v g (%)$
$E D A_(S R) α = 0.4 ， N = 6$	78.3	80.6	79.3
$E D A_s e l f (M D D M) α = 0.4 ， N = 6$	78.8	86.2	82.2
$E D A_e x t e r n a l (M D D M) α = 0.4 ， N = 6$	80.6	84.4	82.4

The Experimental Evaluation of MDDM Entity Replacement Strategy

The Experiment of DSCD Part-of-Speech Replacement Strategy Under Different Parameters

EDA Techniques	$P w - a v g (%)$	$R w - a v g (%)$	$F w - a v g (%)$
$E D A_(R S) α = 0.4 ， N = 10$	78.89	79.75	79.04
$E D A_(D S C D) α = 0.4 ， N = 10$	81.8	85.7	83.6

The Experimental Evaluation of DSCD Part-Speech Replacement Strategy

The Experiment of SPM Random Deletion Strategy Under Different Parameters

DA Techniques	$P w - a v g (%)$	$R w - a v g (%)$	$F w - a v g (%)$
$E D A_(R D) α = 0.6 ， N = 10$	74.4	81.4	77.5
$E D A_(S P M) α = 0.6 ， N = 10$	80.7	84.4	82.0

The Experimental Evaluation of SPM Random Deletion Strategy

The Evaluation of RSI Strategy Under Different Parameters

DA Techniques	$P w - a v g (%)$	$R w - a v g (%)$	$F w - a v g (%)$
$E D A_(R I) α = 0.4 ， N = 10$	80.1	81.8	80.8
$E D A_(R S I) α = 0.4 ， N = 10$	80.9	86.1	83.3

The Experimental Evaluation of RSI Strategy

DA Techniques	$P w - a v g (%)$	$R w - a v g (%)$	$F w - a v g (%)$
$E D A_(M D D M) α = 0.4$	78.8	86.2	82.2
$E D A_(R P S) α = 0.2$	79.1	84.1	81.4
$E D A_(S P M) α = 0.2$	79.4	82.7	80.9
$E D A_(R S I) α = 0.2$	79.7	83.9	81.6
$E D A_(M D D M_R S I) α M D D M = 0.4 ， ? ? α R S I = 0.6$	79.0	86.6	82.4
$? ? E D A_(M D D M_S P M) α S R D = 0.4 ， α S P M = 0.3$	77.8	84.3	80.7
$? E D A_(M D D M_D S C D) α M D D M = 0.4 ， α R P S = 0.2$	79.9	85.1	82.3
$? ? E D A_(D S C D_S P M) α D S C D = 0.2 ， ? α S P M = 0.3$	77.9	83.6	80.4
$? ? ? ? ? E D A_(R S I_R P S_S P M) α R S I = 0.6 ， α R P S = 0.2 ， α S P M = 0.3$	78.0	84.0	80.6
$E D A_(M D D M_R P S_S P M_R S I) α M D D M = 0.4, α R S I = 0.6, α R P S = 0.2, α S P M = 0.3$	80.2	83.2	81.5

The Experimental Evaluation of Multiple Strategy

The Small Sample Dataset of Name Entities Recognition from PeopleDaily

The Small Sample Dataset of Name Entities Recognition from WeiBo

Method	PeopleDaily				WeiBo
Method	S（%）	M（%）	L（%）	F（%）	S（%）	M（%）	L（%）	F（%）
No augmentation	69.7	72.0	78.5	90.9	32.3	37.4	41.8	43.6
SR	70.0	71.4	78.7	88.7	29.0	40.1	43.6	43.8
$M D D M α = 0.4$	72.4	73.4	78.9	90.6	33.45	44.1	43.9	43.2
RS	69.9	67.6	76.3	87.8	33.2	37.7	43.2	41.8
$R P S α = 0.4$	71.4	74.1	79.8	91.3	32.0	36.9	40.3	40.3
RD	69.2	70.5	79.6	90.2	31.9	40.6	41.7	45.3
$S P M α = 0.3$	70.1	72.3	74.3	90.5	35.3	40.6	44.2	45.5
RI	64.5	66.6	73.2	90.9	35.4	42.1	39.7	46.1
$R S I α = 0.4$	71.4	72.2	79.7	91.3	34.9	39.8	42.8	46.2
$E D A_(M D D M_D S C D) α = 0.4 ， 0.2$	73.2	72.5	79.5	90.6	37.7	42.5	38.7	42.7
$E D A_(M D D M_R P S_S P M_R S I)$	70.3	72.2	79.2	91.2	32.8	37.7	43.2	43.9

The Experimental Evaluation of Different Scale Datasets

[1]	邓依依, 邬昌兴, 魏永丰, 等. 基于深度学习的命名实体识别综述[J]. 中文信息学报, 2021, 35(9): 30-45.
[1]	(Deng Yiyi, Wu Changxing, Wei Yongfeng, et al. A Survey on Named Entity Recognition Based on Deep Learning[J]. Journal of Chinese Information Processing, 2021, 35(9): 30-45.)
[2]	Wei J, Zou K. EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks[OL]. arXiv Preprint, arXiv: 1901.11196.
[3]	肖中华. 一种基于群智的语料库数据标注方法及系统:中国,CN108874763A[P]. 2018-11-23[2022-08-14].
[3]	(Xiao Zhonghua. Corpus Data Labeling Method Based on Swarm Intelligence: China, CN108874763A[P]. 2018-11-23[2022-08-14].)
[4]	李贺, 刘嘉宇, 李世钰, 等. 基于疾病知识图谱的自动问答系统优化研究[J]. 数据分析与知识发现, 2021, 5(5): 115-126.
[4]	(Li He, Liu Jiayu, Li Shiyu, et al. Optimizing Automatic Question Answering System Based on Disease Knowledge Graph[J]. Data Analysis and Knowledge Discovery, 2021, 5(5): 115-126.)
[5]	钱力, 谢靖, 常志军, 等. 基于科技大数据的智能知识服务体系研究设计[J]. 数据分析与知识发现, 2019, 3(1): 4-14.
[5]	(Qian Li, Xie Jing, Chang Zhijun, et al. Designing Smart Knowledge Services with SCI-Tech Big Data[J]. Data Analysis and Knowledge Discovery, 2019, 3(1): 4-14.)
[6]	Nadler B, Srebro N, Srebro N, Birch A. Improving Neural Machine Translation Models with Monolingual Data[OL]. arXiv Preprint, arXiv: 1511.06709.
[7]	Park D S, Chan W, Zhang Y, et al. SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition[OL]. arXiv Preprint, arXiv: 1904.08779.
[8]	冯晓硕, 沈樾, 王冬琦. 基于图像的数据增强方法发展现状综述[J]. 计算机科学与应用, 2021(2): 370-382.
[8]	(Feng Xiaoshuo, Shen Yue, Wang Dongqi. A Survey on the Development of Image Data Augmentation[J]. Computer Science and Application, 2021(2): 370-382.)
[9]	黄法秀, 张世杰, 吴志红, 等. 数据增广下的人脸识别研究[J]. 计算机技术与发展, 2020, 30(3): 67-72.
[9]	(Huang Faxiu, Zhang Shijie, Wu Zhihong, et al. Research on Face Recognition Based on Data Augmentation[J]. Computer Technology and Development, 2020, 30(3): 67-72.)
[10]	Shorten C, Khoshgoftaar T M. A Survey on Image Data Augmentation for Deep Learning[J]. Journal of Big Data, 2019, 6: Article No.60.
[11]	Xie Q Z, Dai Z H, Hovy E, et al. Unsupervised Data Augmentation for Consistency Training[OL]. arXiv Preprint, arXiv: 1904.12848.
[12]	张卫, 王昊, 陈玥彤, 等. 融合迁移学习与文本增强的中文成语隐喻知识识别与关联研究[J]. 数据分析与知识发现, 2022, 6(2/3): 167-183.
[12]	(Zhang Wei, Wang Hao, Chen Yuetong, et al. Identifying Metaphors and Association of Chinese Idioms with Transfer Learning and Text Augmentation[J]. Data Analysis and Knowledge Discovery, 2022, 6(2/3): 167-183.)
[13]	刘彤, 刘琛, 倪维健. 多层次数据增强的半监督中文情感分析方法[J]. 数据分析与知识发现, 2021, 5(5): 51-58.
[13]	(Liu Tong, Liu Chen, Ni Weijian. A Semi-Supervised Sentiment Analysis Method for Chinese Based on Multi-Level Data Augmentation[J]. Data Analysis and Knowledge Discovery, 2021, 5(5): 51-58.)
[14]	李健, 张克亮, 唐亮, 等. 面向中文命名实体识别任务的数据增强[J]. 计算机与现代化, 2022(4): 1-6.
[14]	(Li Jian, Zhang Keliang, Tang Liang, et al. Data Augmentation for Chinese Named Entity Recognition Task[J]. Computer and Modernization, 2022(4): 1-6.)
[15]	杨鹤, 于红, 刘巨升, 等. 基于BERT+BiLSTM+CRF深度学习模型和多元组合数据增广的渔业标准命名实体识别[J]. 大连海洋大学学报, 2021, 36(4): 661-669.
[15]	(Yang He, Yu Hong, Liu Jusheng, et al. Fishery Standard Named Entity Recognition Based on BERT+BiLSTM+CRF Deep Learning Model and Multivariate Combination Data Augmentation[J]. Journal of Dalian Ocean University, 2021, 36(4): 661-669.)
[16]	毕佳晶, 李敏, 郑蕊蕊, 等. 面向满文字符识别的训练数据增广方法研究[J]. 大连民族大学学报, 2018, 20(1): 73-78.
[16]	(Bi Jiajing, Li Min, Zheng Ruirui, et al. Research on Training Data Augmentation Methods for Manchu Character Recognition[J]. Journal of Dalian Minzu University, 2018, 20(1): 73-78.)
[17]	王蓬辉, 李明正, 李思. 基于数据增强的中文医疗命名实体识别[J]. 北京邮电大学学报, 2020, 43(5): 84-90. doi: 10.13190/j.jbupt.2020-032
[17]	(Wang Penghui, Li Mingzheng, Li Si. Data Augmentation for Chinese Clinical Named Entity Recognition[J]. Journal of Beijing University of Posts and Telecommunications, 2020, 43(5): 84-90.) doi: 10.13190/j.jbupt.2020-032
[18]	Keraghel A, Benabdeslem K, Canitia B. Data Augmentation Process to Improve Deep Learning-Based NER Task in the Automotive Industry Field[C]// Proceedings of the 2020 International Joint Conference on Neural Networks. IEEE, 2020: 1-8.
[19]	马晓琴, 郭小鹤, 薛峪峰, 等. 针对命名实体识别的数据增强技术[J]. 华东师范大学学报(自然科学版), 2021(5): 14-23.
[19]	(Ma Xiaoqin, Guo Xiaohe, Xue Yufeng, et al. Data Augmentation Technology for Named Entity Recognition[J]. Journal of East China Normal University (Natural Science), 2021(5): 14-23.)
[20]	Dai X, Adel H. An Analysis of Simple Data Augmentation for Named Entity Recognition[OL]. arXiv Preprint, arXiv: 2010.11683.
[21]	Chen J A, Wang Z H, Tian R, et al. Local Additivity Based Data Augmentation for Semi-Supervised NER[OL]. arXiv Preprint, arXiv: 2010.01677.
[22]	刘卫平, 张豹, 陈伟荣, 等. 基于迁移表示学习的军事命名实体识别[J]. 指挥信息系统与技术, 2020, 11(2): 64-69.
[22]	(Liu Weiping, Zhang Bao, Chen Weirong, et al. Military Named Entity Recognition Based on Transfer Representation Learning[J]. Command Information System and Technology, 2020, 11(2): 64-69.)
[23]	徐建, 阮国庆, 李晓冬, 等. 基于迁移学习的小样本军事文本命名实体识别[C]. 见: 第九界中国指挥控制大会论文集. 2021: 288-291.
[23]	(Xu Jian, Ruan Gouqing, Li Xiaodong, et al. Transfer Learning Based Few-Shot Learning for Military Name Entity Recognition[C]// Proceedings of the 9th China Command and Control Conference. 2021:288-291.)
[24]	Yadav V, Sharp R, Bethard S. Deep Affix Features Improve Neural Named Entity Recognizers[C]// Proceedings of the 7th Joint Conference on Lexical and Computational Semantics. 2018: 167-172.
[25]	Sabty C, Omar I, Wasfalla F, et al. Data Augmentation Techniques on Arabic Data for Named Entity Recognition[J]. Procedia Computer Science, 2021, 189: 292-299. doi: 10.1016/j.procs.2021.05.092
[26]	刘焕勇. 开源军事武器装备知识图谱[EB/OL].(2020-04-19). [2022-06-16]. http://openkg.cn/dataset/military-weapon-kg.
[26]	(Liu Huanyong. Open Source Military Weaponry Knowledge Graph[DB/OL]. (2020-04-19). [2022-06-16]. http://openkg.cn/dataset/military-weapon-kg.)
[27]	Dai X, Adel H. An Analysis of Simple Data Augmentation for Named Entity Recognition[C]// Proceedings of the 28th International Conference on Computational Linguistics. 2020: 3861-3867.
[28]	Abid F, Li C, Alam M. Multi-Source Social Media Data Sentiment Analysis Using Bidirectional Recurrent Convolutional Neural Networks[J]. Computer Communications, 2020, 157: 102-115. doi: 10.1016/j.comcom.2020.04.002
[29]	Xie Z, Wang S I, Li J, et al. Data Noising as Smoothing in Neural Network Language Models[OL]. arXiv Preprint, arXiv: 1703.02573.
[30]	Cui Y, Che W, Liu T, et al. Pre-Training with Whole Word Masking for Chinese BERT[OL]. arXiv Preprint, arXiv: 1906.08101.
[31]	Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017: 5998-6008.
[32]	Graves A, Jaitly N, Mohamed A R. Hybrid Speech Recognition with Deep Bidirectional LSTM[C]// Proceedings of the 2013 IEEE Workshop on Automatic Speech Recognition and Understanding. IEEE: 273-278.
[33]	Lafferty J, Mccallum A, Pereira F, et al. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data[C]// Proceedings of the 18th International Conference on Machine Learning. 2001: 282-289.
[34]	马孟铖, 杨晴雯, 艾斯卡尔·艾木都拉, 等. 基于词向量和条件随机场的中文命名实体分类[J]. 计算机工程与设计, 2020, 41(9): 2515-2522.
[34]	(Ma Mengcheng, Yang Qingwen, Askar Hamdulla, et al. Chinese Named Entity Classification Based on Word2Vec and Conditional Random Fields[J]. Computer Engineering and Design, 2020, 41(9): 2515-2522.)
[35]	樊高月, 宫旭平. 美国全球军事基地览要[M]. 第1版. 北京: 解放军出版社, 2014.
[35]	(Fan Gaoyue, Gong Xuping. The Overview of Global U.S. Military Bases[M]. The1st Edition. Beijing: PLA Publishing House, 2014.)

[1]	Jiang Yaren, Le Xiaoqiu. Continual Learning for One-to-many Entity Relationship Generation with Small Samples[J]. 数据分析与知识发现, 2021, 5(8): 45-53.
[2]	Liu Tong,Liu Chen,Ni Weijian. A Semi-Supervised Sentiment Analysis Method for Chinese Based on Multi-Level Data Augmentation[J]. 数据分析与知识发现, 2021, 5(5): 51-58.
[3]	He Xiaojing,Hui Zhibin,Chen Dandan. On the Standard Format of Metadata of E-Government Affairs Archives[J]. 现代图书情报技术, 2003, 19(6): 80-81.

Viewed

Full text

Abstract

Cited

Shared

Discussed