Please wait a minute...
Data Analysis and Knowledge Discovery  2023, Vol. 7 Issue (12): 88-101    DOI: 10.11925/infotech.2096-3467.2022.1080
Current Issue | Archive | Adv Search |
Identifying Chinese Ceramic Genres Based on Image Modal Transfer and Ensemble Learning
Shi Bin1,2,Wang Hao1,2(),Deng Sanhong1,2
1School of Information Management, Nanjing University, Nanjing 210023, China
2Jiangsu Key Laboratory of Data Engineering and Knowledge Service, Nanjing 210023, China
Download: PDF (4279 KB)   HTML ( 9 ) 
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper constructs a clique recognition model for Chinese ceramic images. It aims to automatically classify and recognize the clique of ceramic images and provide technical support for the research and digital protection of ceramic culture. [Methods] We adopted the “end-to-end learning” paradigm to build the new model. It applied transfer learning and ensemble learning technology to ceramic cliff identification. We also used the DCGAN algorithm to balance samples. We examined the new model with ten cliques of ceramics based on their types, crafts, and artistic styles. [Results] The proposed model could more effectively extract ceramic image features and recognize ceramic cliques than the baseline models with manually designed feature engineering. Transfer learning enables the extracted features to be effectively transferred to the fine-grained downstream tasks. The accuracy of the new model reached 73.16%. The improved Stacking method integrated knowledge from the proposed models and increased the final accuracy to 81.39%. [Limitations] The data used in this paper is from Baidu pictures, which need to be expanded to improve the model’s performance. [Conclusions] The new model could effectively classify and identify ceramic images.
Key words： Digital Humanities      Image Recognition      End-to-End Learning      Transfer Learning      Ensemble Learning     
Received: 16 October 2022      Published: 22 March 2023
ZTFLH:  TP391  
  G255  
Fund:National Natural Science Foundation of China(72074108);Jiangsu Provincial Library Society in 2022(22YB056)
Corresponding Authors: Wang Hao，ORCID：0000-0002-0131-0823，E-mail：ywhaowang@nju.edu.cn。   
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
Bin Shi
Hao Wang
Sanhong Deng

Cite this article:

Shi Bin, Wang Hao, Deng Sanhong. Identifying Chinese Ceramic Genres Based on Image Modal Transfer and Ensemble Learning. Data Analysis and Knowledge Discovery, 2023, 7(12): 88-101.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2022.1080     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2023/V7/I12/88

The Experimental Framework
派系标号 名称 数量
1 青花瓷 1 886
2 粉彩瓷 1 778
3 玲珑瓷 1 331
4 颜色釉瓷 1 626
5 其他瓷器 薄胎瓷 1 469
雕塑瓷 1 852
6 紫陶 1 848
7 紫砂陶 1 917
8 桂陶 813
9 安陶 1 638
10 其他陶器 黑陶 1 842
牙舟陶 911
合计 18 911
Distribution of Ceramic Images in Original Data Set
派系
标号		 名称 原始
数据		 清洗后
数据		 训练集
数据		 测试集
数据
1 青花瓷 1 886 952 856 96
2 粉彩瓷 1 778 1 533 1 379 154
3 玲珑瓷 1 331 1 203 1 082 121
4 颜色釉瓷 1 626 1 287 1 158 129
5 其他瓷器 3 321 1 252 1 119 133
6 紫陶 1 848 1 326 1 193 133
7 紫砂陶 1 917 1 747 1 572 175
8 桂陶 813 271 243 28
9 安陶 1 638 371 333 38
10 其他陶器 2 753 947 869 78
合计 18 911 10 889 9 804 1 085
The Training Set and Test Set After Cleaning
Output of Generator of DCGAN in Different Epoches
派系标号 名称 原训练集数量 生成样本数量 合计样本数
1 青花瓷 856 256 1 112
8 桂陶 243 256 499
9 安陶 333 192 525
The Number of Samples Generated by the Generator of DCGAN
T Values on Smoothing Effect of Softmax-T
">
Different T Values on Smoothing Effect of Softmax-T
Training of Baseline Model
Confusion Matrix on Test Set of Baseline Model
Classification Effect of Baseline Model in Specific Ceramic Category
派系标号 Fold1 Fold2 Fold3 Fold4 Fold5 Fold6 Fold7 Fold8 Fold9 Fold10 合计
1 112 112 111 111 111 111 111 111 111 111 1 112
2 138 138 138 138 138 138 138 138 138 137 1 379
3 109 109 108 108 108 108 108 108 108 108 1 082
4 116 116 116 116 116 116 116 116 115 115 1 158
5 112 112 112 112 112 112 112 112 112 111 1 119
6 120 120 120 119 119 119 119 119 119 119 1 193
7 158 158 157 157 157 157 157 157 157 157 1 572
8 50 50 50 50 50 50 50 50 50 49 499
9 53 53 53 53 53 52 52 52 52 52 525
10 87 87 87 87 87 87 87 87 87 86 869
合计 1 055 1 055 1 052 1 051 1 051 1 050 1 050 1 050 1 049 1 045 10 508
Division of 10-Fold Cross Validation in Training Data Set
Classification Effect of End-to-End ResNet50 in Specific Ceramic Category
CAM Heat Maps of Some Test Cases
ResNet50/101/152’s Classification Effect on Various Ceramic Classes
编号 模型 模型
大小		 准确率 平均
查全率		 平均
查准率		 平均
F1
1 ResNet50 269.7MB 0.680 3 0.616 6 0.640 1 0.612 2
2 ResNet101 465.4MB 0.718 1 0.656 0 0.696 9 0.659 8
3 ResNet152 632.2MB 0.722 3 0.658 3 0.697 6 0.660 7
4 DenseNet121 81.6MB 0.731 6 0.683 0 0.716 4 0.682 0
5 Xception 233.8MB 0.716 1 0.644 8 0.709 6 0.645 2
Comparison in Various Indicators of Models
Classification Effect of DenseNet121 in Specific Ceramic Category
序号 次级学习器 准确率 平均查全率 平均查准率 平均F1值 桂陶F1值 安陶F1值
1 LR 0.806 5 0.761 3 0.801 4 0.770 1 0.571 4 0.588 2
2 DT 0.612 9 0.569 0 0.559 5 0.560 9 0.181 8 0.333 3
3 NN 0.763 1 0.709 7 0.743 7 0.713 1 0.391 3 0.567 2
The Classification Results with Different Secondary Learners
序号 初级学习器输出形式 准确率 平均查全率 平均查准率 平均F1值 桂陶F1值 安陶F1值
1 Softmax 0.806 5 0.761 3 0.801 4 0.770 1 0.571 4 0.588 2
2 GAP 0.791 7 0.733 0 0.792 4 0.736 8 0.342 9 0.588 2
3 Logit 0.809 2 0.776 3 0.793 4 0.780 4 0.590 9 0.628 6
4 Softmax-T （T=2 ） 0.811 0 0.770 2 0.801 6 0.777 5 0.612 2 0.608 7
The Classification Results with Different Forms of Primary Learners’ Output
序号 T值 准确率 平均
查全率		 平均
查准率		 平均
F1值		 桂陶
F1值		 安陶
F1值
1 2 0.811 0 0.770 2 0.801 6 0.777 5 0.590 9 0.608 7
2 3 0.810 1 0.776 0 0.793 0 0.779 7 0.612 2 0.608 6
3 5 0.813 9 0.777 8 0.794 3 0.781 4 0.612 2 0.608 7
4 10 0.809 2 0.775 8 0.792 9 0.778 6 0.595 7 0.638 9
5 20 0.801 8 0.757 1 0.788 0 0.763 4 0.511 6 0.617 6
The Classification Results with Different T Values of Softmax-T
[1] 薛翔, 马海云, 张轩慧, 等. 时代经纬:迈向新文科的数字人文——第三届中国数字人文大会综述[J]. 图书馆杂志, 2022, 41(9): 95-104.
[1] (Xue Xiang, Ma Haiyun, Zhang Xuanhui, et al. Digital Humanities Towards New Liberal Arts: A Systematic Review of the Third China Digital Humanities Conference[J]. Library Journal, 2022, 41(9): 95-104.)
[2] 马行云. 中国四大名陶[M]. 昆明: 云南美术出版社, 2014.
[2] (Ma Xingyun. Top Four Chinese Potteries[M]. Kunming: Yunnan Fine Art Publishing House, 2014.)
[3] 曾蕾, 王晓光, 范炜. 图档博领域的智慧数据及其在数字人文研究中的角色[J]. 中国图书馆学报, 2018, 44(1): 17-34.
[3] (Zeng Marcia Lei, Wang Xiaoguang, Fan Wei. Smart Data from Libraries, Archives and Museums and Its Role in the Digital Humanity Researches[J]. Journal of Library Science in China, 2018, 44(1): 17-34.)
[4] 孙静, 李文华. 运用数字人文技术推进非遗保护[N]. 河北经济日报, 2021-12-04(3).
[4] (Sun Jing, Li Wenhua. Using Digital Humanities Technology to Promote the Protection of Intangible Cultural Heritage[N]. Hebei Economic Daily, 2021, 12(4): 3-4.)
[5] 邱兴盛, 邓兴智, 赖文勇, 等. 机器视觉系统在瓷砖缺陷检测中的应用[J]. 陶瓷, 2021(1): 34-41.
[5] (Qiu Xingsheng, Deng Xingzhi, Lai Wenyong, et al. Application of Machine Vision System in Defect Detection of Ceramic Tile[J]. Ceramics, 2021(1): 34-41.)
[6] 陈潇, 徐曙, 张成巍, 等. 基于改进U-Net网络对输电塔陶瓷绝缘子表面缺陷的识别算法[J]. 陶瓷学报, 2022, 43(3): 469-477.
[6] (Chen Xiao, Xu Shu, Zhang Chengwei, et al. Identification Algorithm of Ceramic Insulator Surface Defects on Power Transmission Towers Based on Improved U-Net Network[J]. Journal of Ceramics, 2022, 43(3): 469-477.)
[7] 朱永红, 杨建. 基于深度学习对陶瓷梭式窑火焰图像识别[J]. 西部皮革, 2018, 40(22): 152-153.
[7] (Zhu Yonghong, Yang Jian. Flame Image Recognition of Ceramic Shuttle Kiln Base on Deep Learning[J]. West Leather, 2018, 40(22): 152-153.)
[8] 付瑶. 基于深度学习陶瓷梭式窑烧结工况火焰图像识别方法研究[D]. 景德镇: 景德镇陶瓷大学, 2022.
[8] (Fu Yao. Research on Flame Image Recognition Method of Ceramic Shuttle Kiln Sintering Conditions Based on Deep Learning[D]. Jingdezhen: Jingdezhen Ceramic University, 2022.)
[9] 张静, 周强, 王莹, 等. 元明清陶瓷蕉叶纹纹饰的特征[J]. 硅酸盐学报, 2020, 48(9): 1373-1381.
[9] (Zhang Jing, Zhou Qiang, Wang Ying, et al. Characteristics of Banana Leaf Pattern for Ceramics in Yuan, Ming and Qing Dynasties[J]. Journal of the Chinese Ceramic Society, 2020, 48(9): 1373-1381.)
[10] Mahale V H, Ali M M H, Yannawar P L, et al. Image Inconsistency Detection Using Local Binary Pattern(LBP)[J]. Procedia Computer Science, 2017, 115: 501-508.
doi: 10.1016/j.procs.2017.09.097
[11] 夏静文. 支持向量机(SVM)在古陶瓷分类中的应用研究[D]. 景德镇: 景德镇陶瓷大学, 2022.
[11] (Xia Jingwen. Research on the Application of Support Vector Machine(SVM) in Ancient Ceramic Classification[D]. Jingdezhen: Jingdezhen Ceramic University, 2022.)
[12] 穆天红. 基于人工智能的古陶瓷器型和纹饰图像特征识别研究[D]. 西安: 陕西科技大学, 2020.
[12] (Mu Tianhong. Image Feature Recognition of Ancient Ceramic Shapes and Ornamentation Based on Artificial Intelligence[D]. Xi’an: Shaanxi University of Science & Technology, 2020.)
[13] He M, Feng L, Fan J M. A Method for Real-Time Monitoring of Running Trains Using Ф-OTDR and the Improved Canny[J]. Optik, 2019, 184: 356-363.
doi: 10.1016/j.ijleo.2019.04.112
[14] Zhao L J, Bai H H, Wang A H, et al. Two-Stage Filtering of Compressed Depth Images with Markov Random Field[J]. Signal Processing: Image Communication, 2017, 54: 11-22.
doi: 10.1016/j.image.2017.02.009
[15] Alvarado-Gonzalez M, Aguilar W, Garduño E, et al. Mirror Symmetry Detection in Curves Represented by Means of the Slope Chain Code[J]. Pattern Recognition, 2019, 87: 67-79.
doi: 10.1016/j.patcog.2018.10.002
[16] Stricker M A, Orengo M. Similarity of Color Images[C]// Proceedings the International Society for Optical Engineering. 1995. DOI:10.1117/12.205308.
[17] 张洪波, 詹胜, 高伟, 等. 基于3D点云的卫生陶瓷坯型识别系统设计与实现[J]. 唐山师范学院学报, 2022, 44(3): 63-65.
[17] (Zhang Hongbo, Zhan Sheng, Gao Wei, et al. The Design and Implementation of Sanitary Ware Parison Recognition System Based on 3D Point Cloud[J]. Journal of Tangshan Normal University, 2022, 44(3): 63-65.)
[18] 李颜瑞. 基于PCA和SVM的陶瓷零件研磨后的正废品识别[J]. 信息记录材料, 2021, 22(5): 8-10.
[18] (Li Yanrui. Ceramic Parts After Grinding Recognition Based on PCA and SVM[J]. Information Recording Materials, 2021, 22(5): 8-10.)
[19] 张志林, 李玉鑑, 刘兆英, 等. 深度学习在细粒度图像识别中的应用综述[J]. 北京工业大学学报, 2021, 47(8): 942-953.
[19] (Zhang Zhilin, Li Yujian, Liu Zhaoying, et al. Deep Learning for Fine-Grained Image Recognition: A Survey[J]. Journal of Beijing University of Technology, 2021, 47(8): 942-953.)
[20] 周飞燕, 金林鹏, 董军. 卷积神经网络研究综述[J]. 计算机学报, 2017, 40(6): 1229-1251.
[20] (Zhou Feiyan, Jin Linpeng, Dong Jun. Review of Convolutional Neural Network[J]. Chinese Journal of Computers, 2017, 40(6): 1229-1251.)
[21] Krizhevsky A, Sutskever I, Hinton G E. ImageNet Classification with Deep Convolutional Neural Networks[C]// Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1. ACM, 2012: 1097-1105.
[22] Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition[OL]. arXiv Preprint, arXiv: 1409.1556.
[23] He K M, Zhang X Y, Ren S Q, et al. Deep Residual Learning for Image Recognition[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2016: 770-778.
[24] Huang G, Liu Z, van der Maaten L, et al. Densely Connected Convolutional Networks[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2017: 2261-2269.
[25] Chollet F. Xception: Deep Learning with Depthwise Separable Convolutions[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2017: 1800-1807.
[26] Howard A G, Zhu M L, Chen B, et al. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications[OL]. arXiv Preprint, arXiv: 1704.04861.
[27] Radford A, Metz L, Chintala S. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks[OL]. arXiv Preprint, arXiv: 1511.06434.
[28] Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative Adversarial Networks[J]. Communications of the ACM, 2014, 63: 139-144.
doi: 10.1145/3422622
[29] 曾铭杰. 基于卷积神经网络的陶瓷图像识别及其应用[D]. 景德镇: 景德镇陶瓷大学, 2021.
[29] (Zeng Mingjie. Ceramic Image Recognition and Its Application Based on Convolutional Neural Network[D]. Jingdezhen: Jingdezhen Ceramic University, 2021.)
[30] 梁亚星, 王秀峰, 金晓雪. 基于纹饰最小构成单元特征标记的古陶瓷人工智能识别方法[J]. 中国陶瓷, 2022, 58(7): 64-71.
[30] (Liang Yaxing, Wang Xiufeng, Jin Xiaoxue. Artificial Intelligence Recognition Method of Ancient Ceramics Based on the Feature Mark of the Smallest Structural Unit Pattern[J]. China Ceramics, 2022, 58(7): 64-71.)
[31] 赵凯琳, 靳小龙, 王元卓. 小样本学习研究综述[J]. 软件学报, 2021, 32(2): 349-369.
[31] (Zhao Kailin, Jin Xiaolong, Wang Yuanzhuo. Survey on Few-Shot Learning[J]. Journal of Software, 2021, 32(2): 349-369.)
[32] 郑宇. 基于深度学习的样本不平衡问题研究[D]. 重庆: 重庆理工大学, 2022.
[32] (Zheng Yu. Research on Sample Imbalance Problem Based on Deep Learning[D]. Chongqing: Chongqing University of Technology, 2022.)
[33] Pan S J, Yang Q. A Survey on Transfer Learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22: 1345-1359.
doi: 10.1109/TKDE.2009.191
[34] 吴轶华. 集成学习[J]. 计算机研究与发展, 1991(9): 1-6.
[34] (Wu Yihua. Ensemble Learning[J]. Journal of Computer Research and Development, 1991(9): 1-6.)
[35] Wolpert D H. Stacked Generalization[J]. Neural Networks, 1992, 5(2): 241-259.
doi: 10.1016/S0893-6080(05)80023-1
[36] 史佳琪, 张建华. 基于多模型融合Stacking集成学习方式的负荷预测方法[J]. 中国电机工程学报, 2019, 39(14): 4032-4042.
[36] (Shi Jiaqi, Zhang Jianhua. Load Forecasting Based on Multi-Model by Stacking Ensemble Learning[J]. Proceedings of the CSEE, 2019, 39(14): 4032-4042.)
[37] Hinton G, Vinyals O, Dean J. Distilling the Knowledge in a Neural Network[OL]. arXiv Preprint, arXiv: 1503.02531.
[38] Zhou B L, Khosla A, Lapedriza À, et al. Learning Deep Features for Discriminative Localization[OL]. arXiv Preprint, arXiv: 1512.04150.
[39] Borisov V, Leemann T, Sessler K, et al. Deep Neural Networks and Tabular Data: A Survey[J]. IEEE Transactions on Neural Networks and Learning Systems, 2022. DOI:10.1109/TNNLS.2022.3229161.
[40] 吴琳, 夏孝言, 吴军明. 景德镇陶瓷非物质文化遗产保护和传承研究——以景德镇传统陶瓷制作工艺为例[J]. 中国陶瓷工业, 2021, 28(6): 68-71.
[40] (Wu Lin, Xia Xiaoyan, Wu Junming. Research on the Protection and Inheritance of Jingdezhen Ceramic Intangible Cultural Heritage—Case Study of Jingdezhen Traditional Ceramic Making Technology[J]. China Ceramic Industry, 2021, 28(6): 68-71.)
[1] Han Pu, Gu Liang, Ye Dongyu, Chen Wenqi. Recognizing Chinese Medical Literature Entities Based on Multi-Task and Transfer Learning[J]. 数据分析与知识发现, 2023, 7(9): 136-145.
[2] Yang Chen, Zheng Ruozhen, Wang Chuhan, Geng Shuang, Wang Nan. Ensemble Factorization Machine and Its Application in Paper Recommendation[J]. 数据分析与知识发现, 2023, 7(8): 128-137.
[3] Yan Ying, Huang Qi, Li Na. Feature Selection and Efficient Disease Early Warning Based on Optimized Ensemble Learning Model：Case Study of Geriatric Depression and Anxiety[J]. 数据分析与知识发现, 2023, 7(7): 74-88.
[4] Han Pu, Zhong Yule, Lu Haojie, Ma Shiwen. Identifying Named Entities of Adverse Drug Reaction with Adversarial Transfer Learning[J]. 数据分析与知识发现, 2023, 7(3): 131-141.
[5] Zhang Yiqin, Deng Sanhong, Hu Haotian, Wang Dongbo. Identifying Styles of Cross-Language Classics with Pre-Trained Models[J]. 数据分析与知识发现, 2023, 7(10): 50-62.
[6] Gao Jinsong, Zhang Qiang, Li Shuaike, Sun Yanling, Zhou Shubin. Poet’s Emotional Trajectory in Time and Space: Case Study of Li Bai for Digital Humanities[J]. 数据分析与知识发现, 2022, 6(9): 27-39.
[7] Chen Guo, Ye Chao. News Classification with Semi-Supervised and Active Learning[J]. 数据分析与知识发现, 2022, 6(4): 28-38.
[8] Yi Xinhe, Yang Peng, Wen Yimin. Cross-domain Transfer Learning for Recognizing Professional Skills from Chinese Job Postings[J]. 数据分析与知识发现, 2022, 6(2/3): 274-288.
[9] Fan Tao, Wang Hao, Li Yueyan, Deng Sanhong. Classifying Images of Intangible Cultural Heritages with Multimodal Fusion[J]. 数据分析与知识发现, 2022, 6(2/3): 329-337.
[10] Zhou Zeyu, Wang Hao, Zhang Xiaoqin, Tao Fao, Ren Qiutong. Classification Model for Chinese Traditional Embroidery Based on Xception-TD[J]. 数据分析与知识发现, 2022, 6(2/3): 338-347.
[11] Zhang Wei, Wang Hao, Chen Yuetong, Fan Tao, Deng Sanhong. Identifying Metaphors and Association of Chinese Idioms with Transfer Learning and Text Augmentation[J]. 数据分析与知识发现, 2022, 6(2/3): 167-183.
[12] Wang Nan, Li Hairong, Tan Shuru. Predicting Public Opinion Reversal Based on Evolution Analysis of Events and Improved KE-SMOTE Algorithm[J]. 数据分析与知识发现, 2022, 6(2/3): 396-408.
[13] Fan Tao, Wang Hao, Zhang Wei, Li Xiaomin. Extracting Entities from Intangible Cultural Heritage Texts Based on Machine Reading Comprehension[J]. 数据分析与知识发现, 2022, 6(12): 70-79.
[14] Che Hongxin,Wang Tong,Wang Wei. Comparing Prediction Models for Prostate Cancer[J]. 数据分析与知识发现, 2021, 5(9): 107-114.
[15] Xu Liangchen, Guo Chonghui. Predicting Survival Rates for Gastric Cancer Based on Ensemble Learning[J]. 数据分析与知识发现, 2021, 5(8): 86-99.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626，82624938   E-mail:jishu@mail.las.ac.cn