1Center for Studies of Information Resources, Wuhan University, Wuhan 430072, China 2School of Information Management, Wuhan University, Wuhan 430072, China 3Big Data Institute, Wuhan University, Wuhan 430072, China
[Objective] To inherit and promote seal culture and enhance the recognition of seals in complex scenarios, this study structurally displays the recognition results and related knowledge using knowledge graphs and visualization techniques. [Methods] We proposed a deep learning model integrating multiple features. First, we extracted the seal images’ color, edge, and grayscale feature maps. Then, we input these feature maps into the deep learning model for recognition. Finally, we compared the recognition results with the nodes in the knowledge graph and visualized the related knowledge. [Results] The study collected and annotated seals from 13 calligraphy and painting works, including “The Cold Food Observance”, with two selected works as the test set. Compared with the VGG16 model, our new model’s precision (P), recall (R), and F1 score improved by 28.40%, 28.67%, and 28.54%, respectively. Without integrating multiple features, the P, R, and F1 values decreased by 24.30%, 20.16%, and 22.74%, respectively. [Limitations] The proposed model can only extract and recognize global features of seals, lacking the ability to identify and infer their local semantic information. [Conclusions] The proposed method has a good effect on seal recognition tasks, where multi-dimensional feature maps can enhance the model’s recognition ability and robustness in complex cases.
张志剑, 夏苏迪, 刘政昊. 融合多特征深度学习的印章识别及应用研究*[J]. 数据分析与知识发现, 2024, 8(3): 143-155.
Zhang Zhijian, Xia Sudi, Liu Zhenghao. Seal Recognition and Application Based on Multi-feature Fusion Deep Learning. Data Analysis and Knowledge Discovery, 2024, 8(3): 143-155.
(Zhang Li, Liu Ruiqin, Zhang Fengyuan. Research on Seal Extraction and Repair in Seal Recognition[J]. China Science and Technology Information, 2008(17): 125-126.)
(Fan Tao, Wang Hao, Chen Yuetong. Research on Multimodal Named Entity Recognition of Local History Based on Deep Transfer Learning[J]. Journal of the China Society for Scientific and Technical Information, 2022, 41(4): 412-423.)
(He Xiaoli, Niu Jiaming. Application of 3D Digital Technology in the Intangible Cultural Heritage Protection: Taking Zhaoqing Duan Ink Stone as the Example[J]. Hundred Schools in Arts, 2016, 32(3): 231-233.)
(Zhao Xueqin, Li Tiane, Zeng Gang. Research on Knowledge Meta Ontology Construction and Association Display for Digital Humanities Image Resources[J]. Information Studies: Theory & Application, 2022, 45(9): 180-187.)
doi: 10.16353/j.cnki.1000-7490.2022.09.024
(Xu Miaomiao, Shao Bo. The Context, Problems and Enlightenment of Digital Humanities Development in China[J]. Research on Library Science, 2020(14): 2-10.)
(Zhang Hongyan, Chen Liping. Research Progress and Enlightenment of International Digital Humanities Based on Library Field[J]. Library Work and Study, 2022(7): 27-33.)
(Huang Shuiqing, Zhou Hao, Peng Qiuru, et al. Automatic Recognition and Bibliometric Analysis of Cited Books[J]. Journal of the China Society for Scientific and Technical Information, 2021, 40(12): 1325-1337.)
(Deng Jun, Wang Ruan. Research on Knowledge Map and Multidimensional Knowledge Discovery of Oral History Archives Resources[J]. Library and Information Service, 2022, 66(7): 3-16.)
(Zhang Qi, Wang Dongbo, Huang Shuiqing, et al. Automatic Reorganization of Historical Records from Time Dimension: From the Perspective of Digital Humanities[J]. Documentation, Information & Knowledge, 2022, 39(1): 51-60, 147.)
(Song Xueyan, Zhong Wenmin. Character Relationship Mining and Visualization of Tan Yankai’s Diary from the Perspective of Digital Humanities[J]. Information Science, 2022, 40(6): 25-35.)
(Zeng Ziming, Sun Shouqiang. Research on Mobile Visual Search Model for Dunhuang Murals[J]. Information and Documentation Services, 2021, 42(2): 104-112.)
(Chen Tao, Li Hui, Zhang Yongjuan, et al. LIBRA Technology Theory and Its Application in Historical Image Resources[J]. Journal of Academic Libraries, 2022, 40(4): 64-74.)
(Situ Lingyun, Shi Jin, Yang Haiping, et al. Construction of Evidence Chain System for Rights Protection in the South China Sea Based on Multi-Modal Knowledge Graph[J]. Journal of Intelligence, 2021, 40(12): 23-29, 44.)
[14]
Oliveira S A, Kaplan F, Lenardo I D. Machine Vision Algorithms on Cadaster Plans[C]// Proceedings of the 12th Annual International Conference of the Alliance of Digital Humanities Organizations. 2017.
(Cui Bin, Wang Dongbo, Huang Shuiqing. The Analysis of Time Distribution and Evolution Characteristics of Crops in Classics: Taking Shihuozhi as an Example[J]. Library and Information Service, 2021, 65(14): 90-100.)
doi: 10.13266/j.issn.0252-3116.2021.14.011
(Peng Qiuru, Fan Wenjie, Huang Shuiqing. Errors Revision and Compilation Quality Evaluation of Index to Chunqiu Jingzhuan in a Digital Background[J]. Library and Information Service, 2021, 65(23): 50-57.)
doi: 10.13266/j.issn.0252-3116.2021.23.006
(Song Xueyan, Huo Xiaonan, Liu Yinpeng, et al. Analysis of the Time-Space Trajectory of Relegated Poets in Complete Collection of Tang Poetry from the Perspective of Digital Humanities[J]. Library and Information Service, 2022, 66(7): 26-34.)
doi: 10.13266/j.issn.0252-3116.2022.07.003
(Chen Tao, Zhang Jing, Zhao Yuxiang, et al. Exploration on the Realization Mechanism of Linked Data of Special Collections in Digital Humanities Practice Case Study of Local Chronicles[J]. Information Studies: Theory & Application, 2022, 45(7): 180-187, 147.)
doi: 10.16353/j.cnki.1000-7490.2022.07.025
(Liu Zhenghao, Ni Zhenni, Gui Wenxuan, et al. Multi-Characteristic Analysis and Knowledge System Construction of Lao Zi “Taoism” Study from the Perspective of Knowledge Association[J]. Journal of Modern Information, 2022, 42(7): 42-56, 104.)
doi: 10.3969/j.issn.1008-0821.2022.07.004
(Zhao Xueqin, Li Tiane. Research on Ontology Construction of Archives Field Oriented to Digital Humanities: Take Tea Road Archives as an Example[J]. Information Studies: Theory & Application, 2022, 45(8): 154-161.)
(Zhang Qi, Wang Dongbo, Huang Shuiqing, et al. Multi-Dimensional Knowledge Reorganization and Visualization of History Books: Based on Records of the Grand Historian[J]. Journal of the China Society for Scientific and Technical Information, 2022, 41(2): 130-141.)
(Mou Jiajun, Wang Jian, He Yuqing, et al. Automatic Positioning Algorithm of Ancient Chinese Painting Seal[J]. Modern Electronics Technique, 2015, 38(2): 96-99, 103.)
(Yang Qin, Ding Li, Jiang Peng, et al. Study on Extraction of Blurred Seals in Calligraphy and Painting Based on Hyperspectral Imaging Technology[J]. Journal of National Museum of China, 2022(7): 136-147.)
(Kang Yaqi, Sun Peng, Lang Yubo, et al. Adaptive Canny Detection of Obsolete Seals in Reconstructed Color Space[J]. Computer Simulation, 2023, 40(8): 230-234, 402.)
(Ge Huaidong, Shang Hong. Discussion on the Extraction of Seal Prints in Ancient Document Images[J]. Computer Applications and Software, 2017, 34(5): 189-194.)
(Li Xinli, Zou Changming, Yang Guotian, et al. SealGAN: Research on the Seal Elimination Based on Generative Adversarial Network[J]. Acta Automatica Sinica, 2021, 47(11): 2614-2622.)
(Yang You, Zhang Ruhui, Xu Pengcheng, et al. Improved U-Net for Seal Segmentation of Republican Archives[J]. Journal of Computer Applications, 2023, 43(3): 943-948.)
(Chen Yaya, Liu Quanxiang, Wang Kaili, et al. Historical Chinese Seal Text Recognition Based on ResNet and Transfer Learning[J]. Computer Engineering and Applications, 2022, 58(10): 125-131.)
doi: 10.3778/j.issn.1002-8331.2101-0247
(Ouyang Huan, Fan Dazhao, Li Dongzi. Invoice Seal Identification Based on Multi-Feature Fusion Decision[J]. Computer Engineering and Design, 2018, 39(9): 2842-2847.)
[31]
Canny J. A Computational Approach to Edge Detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986, 8(6): 679-698.
pmid: 21869365
(Huang Mengtao, Lian Yixin. Lithium Battery Electrode Plate Surface Defect Detection Based on Improved Canny Operator[J]. Chinese Journal of Scientific Instrument, 2021, 42(10): 199-209.)
[33]
Nandini D U, Leni E S. Efficient Shadow Detection by Using PSO Segmentation and Region-Based Boundary Detection Technique[J]. The Journal of Supercomputing, 2019, 75(7): 3522-3533.
doi: 10.1007/s11227-018-2292-y
(Lin Jingdong, Wu Xinyi, Chai Yi, et al. Structure Optimization of Convolutional Neural Networks: A Survey[J]. Acta Automatica Sinica, 2020, 46(1): 24-37.)
[36]
Krizhevsky A, Sutskever I, Hinton G E. ImageNet Classification with Deep Convolutional Neural Networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
doi: 10.1145/3065386
[37]
Simonyan K, Andrew Z. Very Deep Convolutional Networks for Large-Scale Image Recognition[C]// Proceedings of the 3rd International Conference on Learning Representations. Washington: ICLR, 2015:1-14.
[38]
Szegedy C, Liu W, Jia Y Q, et al. Going Deeper with Convolutions[C]// Proceedings of the 28th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 1-9.
[39]
Huang G, Liu Z, van der Maaten L, et al. Densely Connected Convolutional Networks[C]// Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 2261-2269.
(Zhang Zhijian, Liu Zhenghao, Ma Feicheng. Enterprise Risk Identification for Internet Public Opinion Events——Based on KGANN Model[J]. Frontiers of Science and Technology of Engineering Management, 2022, 41(1): 65-73.)
[41]
Xu B, Xu Y, Liang J Q, et al. CN-DBpedia: A Never-Ending Chinese Knowledge Extraction System[C]// Proceedings of the 30th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems. Cham: Springer, 2017: 428-438.
