Please wait a minute...
Data Analysis and Knowledge Discovery  2022, Vol. 6 Issue (2/3): 338-347    DOI: 10.11925/infotech.2096-3467.2021.0909
Current Issue | Archive | Adv Search |
Classification Model for Chinese Traditional Embroidery Based on Xception-TD
Zhou Zeyu1,2,Wang Hao1,2(),Zhang Xiaoqin3,Tao Fao1,2,Ren Qiutong1,2
1School of Information Management, Nanjing University, Nanjing 210023, China
2Jiangsu Key Laboratory of Data Engineering and Knowledge Service, Nanjing 210023, China
3Jinling Library, Nanjing 210023, China
Download: PDF (2579 KB)   HTML ( 5
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper introduces artificial intelligence methods to the field of digital humanities, aiming to address the issues of small data sets, insufficient image feature representation, and low recognition accuracy facing traditional Chinese embroidery image classification. It also tries to prvovide methodology support to the digitalization of intangible cultural heritage protection. [Methods] We utilized deep learning techniques to analyze the embroidery images, and extracted their features. Then, we fine-tuned the Xception model with the migration learning approach, and constructed a Xception-TD method to classify traditional Chinese embroidery. Finally, we explored the impacts of the number and dimensions of fully connected layers, as well as the value of dropouts on the model’s performance. [Results] We found that increasing the number and dimensions of fully connected layers improved the embroidery image feature representation. The accuracy rate of our new model reached 0.96863, which was better than the benchmark model. In multi-classification tasks, the model’s accuracy was also better than that of the benchmark ones. [Limitations] The experimental data set was only constructed with Baidu images, which had small amount of manual taggings. [Conclusions] The proposed model based on transfer learning could improve the accuracy of embroidery classification.

Key wordsDigital Humanities      Computer Vision      Transfer Learning      Xception     
Received: 25 August 2021      Published: 18 February 2022
ZTFLH:  G202  
Fund:National Natural Science Foundation of China(72074108);Nanjing University Liberal Arts Youth Interdisciplinary Team Project(010814370113)
Corresponding Authors: Wang Hao,ORCID:0000-0002-0131-0823     E-mail: ywhaowang@nju.edu.cn

Cite this article:

Zhou Zeyu, Wang Hao, Zhang Xiaoqin, Tao Fao, Ren Qiutong. Classification Model for Chinese Traditional Embroidery Based on Xception-TD. Data Analysis and Knowledge Discovery, 2022, 6(2/3): 338-347.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2021.0909     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2022/V6/I2/3/338

Xception-TD based Classification Method for Chinese Traditional
名称 总计数据 训练集数量 测试集数量
其他非四大名绣 2 000 1 600 400
粤绣 500 400 100
湘绣 500 400 100
蜀绣 500 400 100
苏绣 500 400 100
合计 4 000 3 200 800
Distribution of the Crawled Chinese Embroidery Image
Some Pictures of Su Embroidery in the Dataset
Concept Map of Transfer Learning
序号 模型 模型介绍 准确率
1 未微调的Xception 使用ImageNet预训练好的Xception直接作为特征提取的模型参数,通过Softmax层对目标数据集中华传统刺绣进行分类。 0.951 07
2 未微调的VGG-19 VGG网络体系结构最初是由Simonyan和Zisserman提出的,其中VGG19主要架构是5个卷积层块与三个全连接层[41] 0.942 28
3 未微调的Xception-SVM 使用ImageNet预训练好的Xception直接作为特征提取的模型参数,通过支持向量机SVM对目标数据集中华传统刺绣进行分类。 0.940 32
Text Classification Results of Different Models
Impact of the Number and Dimensionality of Fully Connected Layers on Classification Effectiveness of the Model
Influence of Dropout on Classification Effectiveness of the Model Under Two Fully Connected Layers
Classification Capability of Model in Specific Embroidery Category
The Same Painting Style of Shu Embroidery and Hunan Embroidery
[1] 苏新宁. 不忘初心、牢记使命展望情报学与情报工作的未来[J]. 科技情报研究, 2019, 1(1):1-12.
[1] ( Su Xinning. Remain True to Our Original Aspiration and Keep Our Mission in Mind Looking to the Future of Intelligence Studies and Work[J]. Scientific Information Research, 2019, 1(1):1-12.)
[2] 赵含笑. 基于深度学习的刺绣图像分类与识别研究[D]. 西宁: 青海师范大学, 2020.
[2] ( Zhao Hanxiao. Classification and Recognition Algorithm of Embroidery Images Based on Deep Learning[D]. Xining: Qinghai Normal University, 2020.)
[3] Sarraf A, Azhdari M, Sarraf S. A Comprehensive Review of Deep Learning Architectures for Computer Vision Applications[J]. American Scientific Research Journal for Engineering, Technology, and Sciences (ASRJETS), 2021, 77(1):1-29.
[4] 范涛, 吴鹏, 曹琪. 基于深度学习的多模态融合网民情感识别研究[J]. 信息资源管理学报, 2020, 10(1):39-48.
[4] ( Fan Tao, Wu Peng, Cao Qi. The Research of Sentiment Recognition of Online Users Based on DNNS Multimodal Fusion[J]. Journal of Information Resources Management, 2020, 10(1):39-48.)
[5] 徐芳, 金小璞. 基于关联数据的文化遗产数字化保护研究综述[J]. 国家图书馆学刊, 2020, 29(4):90-99.
[5] ( Xu Fang, Jin Xiaopu. Literature Review on Digital Protection of Cultural Heritage Based on Linked Data[J]. Journal of the National Library of China, 2020, 29(4):90-99.)
[6] 魏清华, 刘勐. 非物质文化遗产知识库构建: 以甘肃省国家级非遗为例[J]. 图书馆学研究, 2020(6):33-38.
[6] ( Wei Qinghua, Liu Meng. Construction of Intangible Cultural Heritage Knowledge Base: A Case Study of National Intangible Cultural Heritage in Gansu[J]. Research on Library Science, 2020(6):33-38.)
[7] 段晓卿. 虚拟现实在非遗保护中的应用探究[J]. 文化艺术研究, 2020, 13(1):19-23.
[7] ( Duan Xiaoqing. The Application of Virtual Reality to the Protection of Intangible Cultural Heritages[J]. Studies in Culture & Art, 2020, 13(1):19-23.)
[8] 文琴. 图书馆参与非物质文化遗产数字化的政策研究[J]. 图书馆建设, 2019(S1):156-160.
[8] ( Wen Qin. Policy Research on the Participation of Libraries in the Digitalization of Intangible Cultural Heritage[J]. Library Development, 2019(S1):156-160.)
[9] 谈国新, 张立龙. 非物质文化遗产数字化保护与传承刍议[J]. 图书馆, 2019(4):79-84.
[9] ( Tan Guoxin, Zhang Lilong. A Brief Discussion on the Digital Protection and Inheritance of Intangible Cultural Heritage[J]. Library, 2019(4):79-84.)
[10] 魏明珠, 郑荣, 杨竞雄. 基于深度学习的图像检索研究进展[J]. 情报科学, 2021, 39(5):184-192.
[10] ( Wei Mingzhu, Zheng Rong, Yang Jingxiong. Advances on Image Retrieval Based on Deep Learning[J]. Information Science, 2021, 39(5):184-192.)
[11] 张燕咏, 张莎, 张昱, 等. 基于多模态融合的自动驾驶感知及计算[J]. 计算机研究与发展, 2020, 57(9):1781-1799.
[11] ( Zhang Yanyong, Zhang Sha, Zhang Yu, et al. Multi-Modality Fusion Perception and Computing in Autonomous Driving[J]. Journal of Computer Research and Development, 2020, 57(9):1781-1799.)
[12] 王树义, 刘赛, 马峥. 基于深度迁移学习的微博图像隐私分类研究[J]. 数据分析与知识发现, 2020, 4(10):80-92.
[12] ( Wang Shuyi, Liu Sai, Ma Zheng. Microblog Image Privacy Classification with Deep Transfer Learning[J]. Data Analysis and Knowledge Discovery, 2020, 4(10):80-92.)
[13] 吕昊远, 俞璐, 周星宇, 等. 半监督深度学习图像分类方法研究综述[J]. 计算机科学与探索, 2021, 15(6):1038-1048.
[13] ( Lyu Haoyuan, Yu Lu, Zhou Xingyu, et al. Review of Semi-Supervised Deep Learning Image Classification Methods[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(6):1038-1048.)
[14] Lowe D G. Distinctive Image Features from Scale-Invariant Keypoints[J]. International Journal of Computer Vision, 2004, 60(2):91-110.
doi: 10.1023/B:VISI.0000029664.99615.94
[15] Tajeripour F, Saberi M, Fekri-Ershad S. Developing a Novel Approach for Content Based Image Retrieval Using Modified Local Binary Patterns and Morphological Transform[J]. International Arab Journal of Information Technology, 2015, 12(6):574-581.
[16] Yeh C H, Lin M H, Chang P C, et al. Enhanced Visual Attention-Guided Deep Neural Networks for Image Classification[J]. IEEE Access, 2020, 8:163447-163457.
doi: 10.1109/ACCESS.2020.3021729
[17] Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition[OL]. arXiv Preprint,arXiv: 1409.1556.
[18] He K M, Zhang X Y, Ren S Q, et al. Deep Residual Learning for Image Recognition[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2016: 770-778.
[19] Chollet F. Xception: Deep Learning with Depthwise Separable Convolutions[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2017: 1800-1807.
[20] 牛鑫鑫, 孙阿猛, 王钎沣, 等. 基于深度学习的遥感图像分类研究[J]. 激光杂志, 2021, 42(5):10-14.
[20] ( Niu Xinxin, Sun Ameng, Wang Qianfeng, et al. Study on Remote Sensing Image Classification Based on Deep Learning[J]. Laser Journal, 2021, 42(5):10-14.)
[21] Alzubaidi L, Zhang J L, Humaidi A J, et al. Review of Deep Learning: Concepts, CNN Architectures, Challenges, Applications, Future Directions[J]. Journal of Big Data, 2021, 8(1):53.
doi: 10.1186/s40537-021-00444-8 pmid: 33816053
[22] 李赵旭, 宋涛, 葛梦飞, 等. 基于改进Inception模型的乳腺癌病理学图像分类[J]. 激光与光电子学进展, 2021, 58(8):0817001.
[22] ( Li Zhaoxu, Song Tao, Ge Mengfei, et al. Breast Cancer Classification from Histopathological Images Based on Improved Inception Model[J]. Laser & Optoelectronics Progress, 2021, 58(8):0817001.)
[23] 高淑萍, 赵清源, 齐小刚, 等. 改进MobileNet的图像分类方法研究[J]. 智能系统学报, 2021, 16(1):11-20.
[23] ( Gao Shuping, Zhao Qingyuan, Qi Xiaogang, et al. Research on the Improved Image Classification Method of MobileNet[J]. CAAI Transactions on Intelligent Systems, 2021, 16(1):11-20.)
[24] 谢豪, 毛进, 李纲. 基于多层语义融合的图文信息情感分类研究[J]. 数据分析与知识发现, 2021, 5(6):103-114.
[24] ( Xie Hao, Mao Jin, Li Gang. Sentiment Classification of Image-Text Information with Multi-Layer Semantic Fusion[J]. Data Analysis and Knowledge Discovery, 2021, 5(6):103-114.)
[25] 范涛, 吴鹏, 王昊, 等. 基于多模态联合注意力机制的网民情感分析研究[J]. 情报学报, 2021, 40(6):656-665.
[25] ( Fan Tao, Wu Peng, Wang Hao, et al. Sentiment Analysis of Online Users Based on Multimodal Co-Attention[J]. Journal of the China Society for Scientific and Technical Information, 2021, 40(6):656-665.)
[26] 乔思波, 庞善臣, 王敏, 等. 基于残差混合注意力机制的脑部CT图像分类卷积神经网络模型[J]. 电子学报, 2021, 49(5):984-991.
doi: 10.12263/DZXB.20200881
[26] ( Qiao Sibo, Pang Shanchen, Wang Min, et al. A Convolutional Neural Network for Brain CT Image Classification Based on Residual Hybrid Attention Mechanism[J]. Acta Electronica Sinica, 2021, 49(5):984-991.)
doi: 10.12263/DZXB.20200881
[27] 许骞艺, 秦贵和, 孙铭会, 等. 基于改进的ResNeSt驾驶员头部状态分类算法[J]. 吉林大学学报(工学版), 2021, 51(2):704-711.
[27] ( Xu Qianyi, Qin Guihe, Sun Minghui, et al. Classification of Drivers' Head Status Based on Improved ResNeSt[J]. Journal of Jilin University (Engineering and Technology Edition), 2021, 51(2):704-711.)
[28] 陈巧红, 陈翊, 李文书, 等. 多尺度SE-Xception服装图像分类[J]. 浙江大学学报(工学版), 2020, 54(9):1727-1735.
[28] ( Chen Qiaohong, Chen Yi, Li Wenshu, et al. Clothing Image Classification Based on Multi-Scale SE-Xception[J]. Journal of Zhejiang University(Engineering Science), 2020, 54(9):1727-1735.)
[29] Zhe Z, Wang Q H, Xing Y D. Research on Big Data Analysis Technology of Chinese Traditional Culture Yue Embroidery Color Network[J]. Journal of Physics: Conference Series, 2019, 1345(2):022021.
doi: 10.1088/1742-6596/1345/2/022021
[30] 张效娟. 非物质文化遗产的数字化保护与开发: 以青海刺绣艺术为例[J]. 青海社会科学, 2018(3):201-204.
[30] ( Zhang Xiaojuan. Digital Protection and Development of Intangible Cultural Heritage: The Case of Qinghai Embroidery Art[J]. Qinghai Social Sciences, 2018(3):201-204.)
[31] 刘净净, 郭飞, 刘玉. 刺绣图片的计算机智能识别[J]. 电脑知识与技术, 2012, 8(35):8483-8486.
[31] ( Liu Jingjing, Guo Fei, Liu Yu. The Computer Intelligent Recognition of Embroidery Pictures[J]. Computer Knowledge and Technology, 2012, 8(35):8483-8486.)
[32] 龚伟伟. 基于卷积神经网络的刺绣图像检索研究[D]. 西宁: 青海师范大学, 2020.
[32] ( Gong Weiwei. Research on Embroidery Image Retrieval Based on Convolutional Neural Network[D]. Xining: Qinghai Normal University, 2020.)
[33] 杨蕾, 胡慧, 周军. 刺绣针法图样特征点提取及匹配方法研究[J]. 计算机应用研究, 2021, 38(7):2231-2234.
[33] ( Yang Lei, Hu Hui, Zhou Jun. Study on Feature Points Extraction and Matching of Stitch Pattern[J]. Application Research of Computers, 2021, 38(7):2231-2234.)
[34] 赵凯琳, 靳小龙, 王元卓. 小样本学习研究综述[J]. 软件学报, 2021, 32(2):349-369.
[34] ( Zhao Kailin, Jin Xiaolong, Wang Yuanzhuo. Survey on Few-Shot Learning[J]. Journal of Software, 2021, 32(2):349-369.)
[35] 陈虹丽, 刘凌风, 李浩凯, 等. 基于迁移学习的指关节纹识别算法实验设计[J]. 实验技术与管理, 2021, 38(6):81-84.
[35] ( Chen Hongli, Liu Lingfeng, Li Haokai, et al. Experimental Design of Knuckle Pattern Recognition Algorithm Based on Transfer Learning[J]. Experimental Technology and Management, 2021, 38(6):81-84.)
[36] 张德军, 周学成, 杨旭东. 基于图像处理和深度迁移学习的芒果果实病状识别[J]. 华南农业大学学报, 2021, 42(4):113-124.
[36] ( Zhang Dejun, Zhou Xuecheng, Yang Xudong. Recognition of Mango Fruit Diseases Based on Image Processing and Deep Transfer Learning[J]. Journal of South China Agricultural University, 2021, 42(4):113-124.)
[37] 陈德刚, 艾孜尔古丽, 尹鹏博, 等. 基于改进Xception迁移学习的野生菌种类识别研究[J]. 激光与光电子学进展, 2021, 58(8):0810023.
[37] ( Chen Degang, Azragul, Yin Pengbo, et al. Research on Identification of Wild Mushroom Species Based on Improved Xception Transfer Learning[J]. Laser & Optoelectronics Progress, 2021, 58(8):0810023.)
[38] 蒋雨肖, 丁晟春, 吴鹏. 基于BiLSTM-VGG16的多模态信息特征分类研究[J]. 情报理论与实践, 2021, 44(11):180-186.
[38] ( Jiang Yuxiao, Ding Shengchun, Wu Peng. A Study on the Classification of Features of Multi-Modal Information Based on BiLSTM-VGG16[J]. Information Studies: Theory & Application, 2021, 44(11):180-186.)
[39] 李昆仑, 王怡辉, 陈栋, 等. 结合注意力与双线性网络的细粒度图像分类[J]. 小型微型计算机系统, 2021, 42(5):1071-1076.
[39] ( Li Kunlun, Wang Yihui, Chen Dong, et al. Combines Attention with Bilinear Networks for Fine-Grained Image Classification[J]. Journal of Chinese Computer Systems, 2021, 42(5):1071-1076.)
[40] 樊湘鹏, 许燕, 周建平, 等. 基于迁移学习和改进CNN的葡萄叶部病害检测系统[J]. 农业工程学报, 2021, 37(6):151-159.
[40] ( Fan Xiangpeng, Xu Yan, Zhou Jianping, et al. Detection System for Grape Leaf Diseases Based on Transfer Learning and Updated CNN[J]. Transactions of the Chinese Society of Agricultural Engineering, 2021, 37(6):151-159.)
[41] Carvalho T, de Rezende E R S, Alves M T P, et al. Exposing Computer Generated Images by Eye’s Region Classification via Transfer Learning of VGG19 CNN[C]// Proceedings of 2017 16th IEEE International Conference on Machine Learning and Applications. IEEE, 2017: 866-870.
[42] Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: A Simple Way to Prevent Neural Networks from Overfitting[J]. Journal of Machine Learning Research, 2014, 15(1):1929-1958.
[1] Yi Xinhe, Yang Peng, Wen Yimin. Cross-domain Transfer Learning for Recognizing Professional Skills from Chinese Job Postings[J]. 数据分析与知识发现, 2022, 6(2/3): 274-288.
[2] Fan Tao, Wang Hao, Li Yueyan, Deng Sanhong. Classifying Images of Intangible Cultural Heritages with Multimodal Fusion[J]. 数据分析与知识发现, 2022, 6(2/3): 329-337.
[3] Zhang Wei, Wang Hao, Chen Yuetong, Fan Tao, Deng Sanhong. Identifying Metaphors and Association of Chinese Idioms with Transfer Learning and Text Augmentation[J]. 数据分析与知识发现, 2022, 6(2/3): 167-183.
[4] Lu Quan, He Chao, Chen Jing, Tian Min, Liu Ting. A Multi-Label Classification Model with Two-Stage Transfer Learning[J]. 数据分析与知识发现, 2021, 5(7): 91-100.
[5] Zhang Qi,Jiang Chuan,Ji Youshu,Feng Minxuan,Li Bin,Xu Chao,Liu Liu. Unified Model for Word Segmentation and POS Tagging of Multi-Domain Pre-Qin Literature[J]. 数据分析与知识发现, 2021, 5(3): 2-11.
[6] Wang Qian,Wang Dongbo,Li Bin,Xu Chao. Deep Learning Based Automatic Sentence Segmentation and Punctuation Model for Massive Classical Chinese Literature[J]. 数据分析与知识发现, 2021, 5(3): 25-34.
[7] Zhao Yuxiang,Lian Jingwen. Review of Cultural Heritage Crowdsourcing in the Domain of Digital Humanities[J]. 数据分析与知识发现, 2021, 5(1): 36-55.
[8] Liang Jiwen,Jiang Chuan,Wang Dongbo. Chinese-English Sentence Alignment of Ancient Literature Based on Multi-feature Fusion[J]. 数据分析与知识发现, 2020, 4(9): 123-132.
[9] Xu Chenfei, Ye Haiying, Bao Ping. Automatic Recognition of Produce Entities from Local Chronicles with Deep Learning[J]. 数据分析与知识发现, 2020, 4(8): 86-97.
[10] Zhao Ping,Sun Lianying,Tu Shuai,Bian Jianling,Wan Ying. Identifying Scenic Spot Entities Based on Improved Knowledge Transfer[J]. 数据分析与知识发现, 2020, 4(5): 118-126.
[11] Liu Tong,Ni Weijian,Sun Yujian,Zeng Qingtian. Predicting Remaining Business Time with Deep Transfer Learning[J]. 数据分析与知识发现, 2020, 4(2/3): 134-142.
[12] Xiang Fei,Xie Yaotan. Recognition Model of Patient Reviews Based on Mixed Sampling and Transfer Learning[J]. 数据分析与知识发现, 2020, 4(2/3): 39-47.
[13] Liu Liu,Qin Tianyun,Wang Dongbo. Automatic Extraction of Traditional Music Terms of Intangible Cultural Heritage[J]. 数据分析与知识发现, 2020, 4(12): 68-75.
[14] Wang Shuyi,Liu Sai,Ma Zheng. Microblog Image Privacy Classification with Deep Transfer Learning[J]. 数据分析与知识发现, 2020, 4(10): 80-92.
[15] Haici Yang,Jun Wang. Visualizing Knowledge Graph of Academic Inheritance in Song Dynasty[J]. 数据分析与知识发现, 2019, 3(6): 109-116.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn