Please wait a minute...
Data Analysis and Knowledge Discovery  2023, Vol. 7 Issue (2): 108-118    DOI: 10.11925/infotech.2096.3467.2022.1083
Current Issue | Archive | Adv Search |
AEMIA:Extracting Commodity Attributes Based on Multi-level Interactive Attention Mechanism
Su Mingxing,Wu Houyue,Li Jian,Huang Ju,Zhang Shunxiang()
School of Computer Science and Engineering, Anhui University of Science & Technology, Huainan 232001, China
Download: PDF (1146 KB)   HTML ( 22 ) 
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper develops a new model to improve the perception of structural features and correlation between text features, aiming to fully explore the internal semantics and extract attributes. [Methods] First, we extracted the features of text, syntax and part of speech. Then, we merged different features to obtain complete text structure features. Third, we designed a multi-layer interactive attention mechanism, which focuses on the deep correlation between text structural features and text features. Fourth, we adopted bilinear fusion strategy to ensure the information integrity. Finally, we extracted attributes with common classifiers. [Results] We examined the new model with publicly available data sets, and found its extraction accuracy was at least 1.2 percentage point higher than that of the existing methods. [Limitations] The model was insensitive to implicit attribute words, and the performance of the model will be greatly reduced with more than three implicit attribute words in the sentence. [Conclusions] The proposed method can effectively improve the accuracy of commodity attributes extraction.
Key words： Attribute Extraction      Interactive Attention Mechanism      Dependency Relationship      BiGRU      BERT     
Received: 17 October 2022      Published: 28 March 2023
ZTFLH:  TP391  
Fund:Natural Science Foundation of China(62076006);University Synergy Innovation Program of Anhui Province(GXXT-2021-008)
Corresponding Authors: Chang Zhijun，ORCID：0000-0001-9211-8599，E-mail： changzj@mail.las.ac.cn。   
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Mingxing Su
Houyue Wu
Jian Li
Ju Huang
Shunxiang Zhang

Cite this article:

Su Mingxing, Wu Houyue, Li Jian, Huang Ju, Zhang Shunxiang. AEMIA:Extracting Commodity Attributes Based on Multi-level Interactive Attention Mechanism. Data Analysis and Knowledge Discovery, 2023, 7(2): 108-118.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096.3467.2022.1083     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2023/V7/I2/108

AEMIA ： Commodity Attribute Extraction Model
An Example of Context Extraction Based on Dependencies
数据集 训练集 测试集
评论语句 属性词 评论语句 属性词
L-14 3 041 2 358 800 654
R-14 3 045 3 693 800 1 134
R-15 1 315 1 192 685 678
R-16 2 000 1 743 676 622
Experimental Dataset
项目 配置
OS Windows10
CPU Intel core Tmi7-8700kcpu@3.7GHz
GPU Tesla V100 （16GB）
Python 3.6
PyTorch 1.6
Memory 64GB
Experimental Environment
参数 值
BERT模型 BERT-basehttps：//github.com/google-research/bert.
CRF模型 CRF++0.58http：//taku910.github.io/crfpp/.
Transformer层数 12
BERT的隐藏层维度 768
BERT中的注意力头数 12
BatchSize 64
Dropout 0.5
Learning rate 1 × 10 - 5
GRU的隐藏层维度 128
优化器 Adam
多层交互注意力的层数 5
Parameter Settings of AEMIA
模型 L-14 R-14 R-15 R-16
Acc/% F1/% Acc/% F1/% Acc/% F1/% Acc/% F1/%
对比实验组 A1 CRF 78.14 74.01 84.06 82.32 72.10 66.54 74.71 69.67
A2 BiGRU+CRF 79.57 76.92 85.73 82.12 73.22 68.12 75.21 71.21
A3 DE-CNN 84.38 81.16 86.14 83.27 74.15 70.18 77.27 74.27
A4 CIA-CRF 83.19 79.12 86.03 84.09 75.26 70.61 78.09 73.21
A5 DepREm-CRF 84.62 81.09 87.21 83.14 77.73 71.86 79.16 74.24
消融实验组 A6 AEMIA（MIA） 83.02 80.11 86.18 82.10 76.69 69.82 78.96 73.23
A7 AEMIA（POS） 84.73 79.05 87.41 83.07 77.03 70.12 80.27 74.92
A8 AEMIA（DE） 83.82 79.71 87.28 83.19 77.91 70.04 80.18 74.88
A9 AEMIA 86.01 81.20 88.93 84.18 79.12 71.05 81.67 75.03
Experimental data
The Effect of Interactional Attention Layers on Accuracy
模型 P值
A1 CRF 0.081 2
A2 BiGRU+CRF 0.041 2
A3 DE-CNN 0.045 2
A4 CIA-CRF 0.024 7
A5 DepREm-CRF 0.037 8
A9 AEMIA 0.017 5
Significance Scores
[1] 张严, 李天瑞. 面向评论的方面级情感分析综述[J]. 计算机科学, 2020, 47(6): 194-200.
doi: 10.11896/jsjkx.200200127
[1] (Zhang Yan,Li Tianrui. Review of Comment-Oriented Aspect-Based Sentiment Analysis[J]. Computer Science, 2020, 47(6): 194-200.)
doi: 10.11896/jsjkx.200200127
[2] Hu M Q, Liu B. Mining and Summarizing Customer Reviews[C]// Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2004: 168-177.
[3] Blair-Goldensohn S, Hannan K, McDonald R, et al. Building a Sentiment Summarizer for Local Service Reviews[C]// Proceedings of WWW Workshop on NLP in the Information Explosion Era. 2008: 339-348.
[4] Poria S, Cambria E, Ku L W, et al. A Rule-Based Approach to Aspect Extraction from Product Reviews[C]// Proceedings of the 2nd Workshop on Natural Language Processing for Social Media. 2014: 28-37.
[5] Rana T A, Cheah Y N. A Two-Fold Rule-Based Model for Aspect Extraction[J]. Expert Systems with Applications, 2017, 89: 273-285.
doi: 10.1016/j.eswa.2017.07.047
[6] Wang B, Wang H F. Bootstrapping Both Product Features and Opinion Words from Chinese Customer Reviews with Cross-Inducing[C]// Proceedings of the 3rd International Joint Conference on Natural Language Processing. 2008: 289-295.
[7] Titov I, McDonald R. Modeling Online Reviews with Multi-grain Topic Models[C]// Proceedings of the 17th International Conference on World Wide Web. 2008: 111-120.
[8] Jakob N, Gurevych I. Extracting Opinion Targets in a Single- and Cross-Domain Setting with Conditional Random Fields[C]// Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing. 2010: 1035-1045.
[9] Hamdan H, Bellot P, Bechet F. Lsislif: CRF and Logistic Regression for Opinion Target Extraction and Sentiment Polarity Analysis[C]// Proceedings of the 9th International Workshop on Semantic Evaluation. 2015: 753-758.
[10] Gupta D K, Reddy K S, Shweta, et al. PSO-Asent: Feature Selection Using Particle Swarm Optimization for Aspect Based Sentiment Analysis[C]// Proceedings of International Conference on Applications of Natural Language to Information Systems. 2015: 220-233.
[11] 彭春艳, 张晖, 包玲玉, 等. 基于条件随机域的生物命名实体识别[J]. 计算机工程, 2009, 35(22): 197-199.
[11] (Peng Chunyan, Zhang Hui, Bao Lingyu, et al. Biological Named Entity Recognition Based on Conditional Random Fields[J]. Computer Engineering, 2009, 35(22): 197-199.)
[12] Toh Z, Wang W T. DLIREC: Aspect Term Extraction and Term Polarity Classification System[C]// Proceedings of the 8th International Workshop on Semantic Evaluation. 2014: 235-240.
[13] Poria S, Cambria E, Gelbukh A. Aspect Extraction for Opinion Mining with a Deep Convolutional Neural Network[J]. Knowledge-Based Systems, 2016, 108: 42-49.
doi: 10.1016/j.knosys.2016.06.009
[14] 苏丰龙, 谢庆华, 邱继远, 等. 基于深度学习的领域实体属性词聚类抽取研究[J]. 微型机与应用, 2016, 35(1): 53-55, 59.
[14] (Su Fenglong, Xie Qinghua, Qiu Jiyuan, et al. Study on Word Clustering for Attribute Extraction Based on Deep Learning[J]. Microcomputer & Its Applications, 2016, 35(1): 53-55, 59.)
[15] Li X, Lam W. Deep Multi-task Learning for Aspect Term Extraction with Memory Interaction[C]// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017: 2886-2892.
[16] 王仁武, 张文慧. 基于深度学习的隐性评价对象识别方法[J]. 计算机工程, 2019, 45(8): 315-320.
[16] (Wang Renwu, Zhang Wenhui. Implicit Evaluation Object Recognition Method Based on Deep Learning[J]. Computer Engineering, 2019, 45(8): 315-320.)
[17] Yin Y C, Wang C G, Zhang M. PoD: Positional Dependency-Based Word Embedding for Aspect Term Extraction[C]// Proceedings of the 28th International Conference on Computational Linguistics. 2020: 1714-1719.
[18] Phan M H, Ogunbona P O. Modelling Context and Syntactical Features for Aspect-Based Sentiment Analysis[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 3211-3220.
[19] Devlin J, Chang M W, Lee K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, Volume 1 (Long and Short Papers). 2019: 4171-4186.
[20] Cho K, van Merriënboer B, Gulcehre C, et al. Learning Phrase Representations Using RNN Encoder-Decoder for Statistical Machine Translation[C]// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 2014: 1724-1734.
[21] Ma R X, Wang K, Qiu T, et al. Feature-Based Compositing Memory Networks for Aspect-Based Sentiment Classification in Social Internet of Things[J]. Future Generation Computer Systems, 2019, 92(C): 879-888.
[22] Liu G, Guo J B. Bidirectional LSTM with Attention Mechanism and Convolutional Layer for Text Classification[J]. Neurocomputing, 2019, 337: 325-338.
doi: 10.1016/j.neucom.2019.01.078
[23] Tenenbaum J B, Freeman W T. Separating Style and Content with Bilinear Models[J]. Neural Computation, 2000, 12(6): 1247-1283.
pmid: 10935711
[24] Kim J H, On K W, Lim W, et al. Hadamard Product for Low-rank Bilinear Pooling[OL]. arXiv Preprint, arXiv: 1610.04325.
[25] Liu Y. Fine-tune BERT for Extractive Summarization[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 2019: 3546-3551.
[26] Toh Z Q, Su J. NLANGPat SemEval-2016 Task 5: Improving Aspect Based Sentiment Analysis Using Neural Network Features[C]// Proceedings of the 10th International Workshop on Semantic Evaluation. 2016: 282-288.
[27] Xu H, Liu B, Shu L, et al. Double Embeddings and CNN-Based Sequence Labeling for Aspect Extraction[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2:Short Papers). 2018: 592-598.
[28] 尉桢楷, 程梦, 周夏冰, 等. 基于类卷积交互式注意力机制的属性抽取研究[J]. 计算机研究与发展, 2020, 57(11): 2456-2466.
[28] (Yu Zhenkai, Cheng Meng, Zhou Xiabing, et al. Convolutional Interactive Attention Mechanism for Aspect Extraction[J]. Journal of Computer Research and Development, 2020, 57(11): 2456-2466.)
[29] 李成梁, 赵中英, 李超, 等. 基于依存关系嵌入与条件随机场的商品属性抽取方法[J]. 数据分析与知识发现, 2020, 4(5): 54-65.
[29] (Li Chengliang, Zhao Zhongying, Li Chao, et al. Extracting Product Properties with Dependency Relationship Embedding and Conditional Random Field[J]. Data Analysis and Knowledge Discovery, 2020, 4(5): 54-65.)
[30] Dror R, Baumer G, Shlomov S, et al. The Hitchhiker’s Guide to Testing Statistical Significance in Natural Language Processing[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers). 2018: 1383-1392.
[1] Zhao Yiming, Pan Pei, Mao Jin. Recognizing Intensity of Medical Query Intentions Based on Task Knowledge Fusion and Text Data Enhancement[J]. 数据分析与知识发现, 2023, 7(2): 38-47.
[2] Wang Yufei, Zhang Zhixiong, Zhao Yang, Zhang Mengting, Li Xuesi. Designing and Implementing Automatic Title Generation System for Sci-Tech Papers[J]. 数据分析与知识发现, 2023, 7(2): 61-71.
[3] Zhang Siyang, Wei Subo, Sun Zhengyan, Zhang Shunxiang, Zhu Guangli, Wu Houyue. Extracting Emotion-Cause Pairs Based on Multi-Label Seq2Seq Model[J]. 数据分析与知识发现, 2023, 7(2): 86-96.
[4] Shi Yunmei, Yuan Bo, Zhang Le, Lv Xueqiang. IMTS: Detecting Fake Reviews with Image and Text Semantics[J]. 数据分析与知识发现, 2022, 6(8): 84-96.
[5] Wu Jiang, Liu Tao, Liu Yang. Mining Online User Profiles and Self-Presentations: Case Study of NetEase Music Community[J]. 数据分析与知识发现, 2022, 6(7): 56-69.
[6] Zheng Jie, Huang Hui, Qin Yongbin. Matching Similar Cases with Legal Knowledge Fusion[J]. 数据分析与知识发现, 2022, 6(7): 99-106.
[7] Pan Huiping, Li Baoan, Zhang Le, Lv Xueqiang. Extracting Keywords from Government Work Reports with Multi-feature Fusion[J]. 数据分析与知识发现, 2022, 6(5): 54-63.
[8] Xiao Yuejun, Li Honglian, Zhang Le, Lv Xueqiang, You Xindong. Classifying Chinese Patent Texts with Feature Fusion[J]. 数据分析与知识发现, 2022, 6(4): 49-59.
[9] Yang Lin, Huang Xiaoshuo, Wang Jiayang, Ding Lingling, Li Zixiao, Li Jiao. Identifying Subtypes of Clinical Trial Diseases with BERT-TextCNN[J]. 数据分析与知识发现, 2022, 6(4): 69-81.
[10] Ding Shengchun, You Weijing, Wang Xiaoying. Extracting Weapon Attributes Based on Word Completion[J]. 数据分析与知识发现, 2022, 6(2/3): 289-297.
[11] Guo Hangcheng, He Yanqing, Lan Tian, Wu Zhenfeng, Dong Cheng. Identifying Moves from Scientific Abstracts Based on Paragraph-BERT-CRF[J]. 数据分析与知识发现, 2022, 6(2/3): 298-307.
[12] Wang Yongsheng, Wang Hao, Yu Wei, Zhou Zeyu. Extracting Relationship Among Characters from Local Chronicles with Text Structures and Contents[J]. 数据分析与知识发现, 2022, 6(2/3): 318-328.
[13] Zhang Yunqiu, Wang Yang, Li Bocheng. Identifying Named Entities of Chinese Electronic Medical Records Based on RoBERTa-wwm Dynamic Fusion Model[J]. 数据分析与知识发现, 2022, 6(2/3): 242-250.
[14] Shang Lili, Tang Huayun, Wang Yanzhao, Zuo Meiyun. Identifying Actionable Information from Online Reviews[J]. 数据分析与知识发现, 2022, 6(12): 1-12.
[15] Yan Dongmei, He Wenxin, Chen Zhi. Predicting Stock Prices Based on RoBERTa-TCN and Sentimental Characteristics[J]. 数据分析与知识发现, 2022, 6(12): 123-134.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626，82624938   E-mail:jishu@mail.las.ac.cn