Please wait a minute...
Data Analysis and Knowledge Discovery  0, Vol. Issue (): 1-    DOI: 10.11925/infotech.2096-3467.2018.2020.0206
Current Issue | Archive | Adv Search |
Few-shot text classification based on Bi-directional Long-term Attention feature expression Few-shot text classification based on Bi-directional Long-term Attention feature expression
Xu Tongtong, Sun Huazhi, Ma Chunmei, Jiang Lifen, Liu Yichen
(College of Computer and Information Engineering, Tianjin Normal University, Tianjin 300387, China)
Download:
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper studies text classification task in few-shot learning setting and proposes a few-shot text classification model to address the issues of data scarcity and low generalization performance. [Methods] We divide the text classification task into multiple subtasks based on episode training mechanism in meta-learning. We propose a Bi-directional Temporal Convolutional Network(Bi-TCN) to capture the long-term contextual information of the text in each subtask. We propose Bi-directional Long-term Attention Network(BLAN) to capture more discriminative features based on Bi-TCN and multi-head attention mechanism. Neural Tensor Network is used to measure the correlation between query samples and support set of each subtask to achieve few-shot text classification. [Results] Experiments are performed on the ARSC dataset, the experimental results show that the classification accuracy of BLAN model reaches 86.80% in few-shot learning setting, which is improved by 3.68% and 1.17% respectively compared with the existing advanced few-shot text classification models ROBUSTTC-FSL and Induction-Network-Routing. [Limitations] The performance of BLAN on long text is not satisfactory. [Conclusions] BLAN overcomes the issue of data scarcity and captures comprehensive feature information of text, which effectively improves the performance of few-shot text classification.

Key words Few-shot text classification      Attention mechanism      Few-shot learning      Bi-TCN      
Published: 17 July 2020
ZTFLH:  TP393  

Cite this article:

Xu Tongtong, Sun Huazhi, Ma Chunmei, Jiang Lifen, Liu Yichen. Few-shot text classification based on Bi-directional Long-term Attention feature expression Few-shot text classification based on Bi-directional Long-term Attention feature expression . Data Analysis and Knowledge Discovery, 0, (): 1-.

URL:

http://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2018.2020.0206     OR     http://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y0/V/I/1

[1] Huang Lu,Zhou Enguo,Li Daifeng. Text Representation Learning Model Based on Attention Mechanism with Task-specific Information[J]. 数据分析与知识发现, 2020, 4(9): 111-122.
[2] Yin Haoran,Cao Jinxuan,Cao Luzhe,Wang Guodong. Identifying Emergency Elements Based on BiGRU-AM Model with Extended Semantic Dimension[J]. 数据分析与知识发现, 2020, 4(9): 91-99.
[3] Shi Lei,Wang Yi,Cheng Ying,Wei Ruibin. Review of Attention Mechanism in Natural Language Processing[J]. 数据分析与知识发现, 2020, 4(5): 1-14.
[4] Xue Fuliang,Liu Lifang. Fine-Grained Sentiment Analysis with CRF and ATAE-LSTM[J]. 数据分析与知识发现, 2020, 4(2/3): 207-213.
[5] Xu Tongtong,Sun Huazhi,Ma Chunmei,Jiang Lifen,Liu Yichen. Classification Model for Few-shot Texts Based on Bi-directional Long-term Attention Features[J]. 数据分析与知识发现, 2020, 4(10): 113-123.
[6] Yuemin Wu,Ganggui Ding,Bin Hu. Extracting Relationship of Agricultural Financial Texts with Attention Mechanism[J]. 数据分析与知识发现, 2019, 3(5): 86-92.
[7] Zhiyong Tao,Xiaobing Li,Ying Liu,Xiaofang Liu. Classifying Short Texts with Improved-Attention Based Bidirectional Long Memory Network[J]. 数据分析与知识发现, 2019, 3(12): 21-29.
[8] Yuman Li,Zhibo Chen,Fu Xu. Classifying Texts with KACC Model[J]. 数据分析与知识发现, 2019, 3(10): 89-97.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn