|
|
|
| Seal Recognition and Application Based on Multi-feature Fusion Deep Learning |
Zhang Zhijian1,2,3( ),Xia Sudi1,2,3,Liu Zhenghao1,2,3 |
1Center for Studies of Information Resources, Wuhan University, Wuhan 430072, China
2School of Information Management, Wuhan University, Wuhan 430072, China
3Big Data Institute, Wuhan University, Wuhan 430072, China |
|
|
|
|
Abstract [Objective] To inherit and promote seal culture and enhance the recognition of seals in complex scenarios, this study structurally displays the recognition results and related knowledge using knowledge graphs and visualization techniques. [Methods] We proposed a deep learning model integrating multiple features. First, we extracted the seal images’ color, edge, and grayscale feature maps. Then, we input these feature maps into the deep learning model for recognition. Finally, we compared the recognition results with the nodes in the knowledge graph and visualized the related knowledge. [Results] The study collected and annotated seals from 13 calligraphy and painting works, including “The Cold Food Observance”, with two selected works as the test set. Compared with the VGG16 model, our new model’s precision (P), recall (R), and F1 score improved by 28.40%, 28.67%, and 28.54%, respectively. Without integrating multiple features, the P, R, and F1 values decreased by 24.30%, 20.16%, and 22.74%, respectively. [Limitations] The proposed model can only extract and recognize global features of seals, lacking the ability to identify and infer their local semantic information. [Conclusions] The proposed method has a good effect on seal recognition tasks, where multi-dimensional feature maps can enhance the model’s recognition ability and robustness in complex cases.
|
|
Received: 02 January 2023
Published: 12 April 2024
|
|
|
| Fund:National Natural Science Foundation of China(91646206);Scientific and Technological Innovation 2030 - “New Generation Artificial Intelligence” Major Project(2020AAA0108505) |
|
Corresponding Authors:
Zhang Zhijian，ORCID：0000-0002-7758-9277，E-mail：zzjian@whu.edu.cn。
|
| [1] |
张栗, 刘瑞芹, 张凤元. 印鉴识别中印章的提取和修复问题研究[J]. 中国科技信息, 2008(17): 125-126.
|
| [1] |
(Zhang Li, Liu Ruiqin, Zhang Fengyuan. Research on Seal Extraction and Repair in Seal Recognition[J]. China Science and Technology Information, 2008(17): 125-126.)
|
| [2] |
范涛, 王昊, 陈玥彤. 基于深度迁移学习的地方志多模态命名实体识别研究[J]. 情报学报, 2022, 41(4): 412-423.
|
| [2] |
(Fan Tao, Wang Hao, Chen Yuetong. Research on Multimodal Named Entity Recognition of Local History Based on Deep Transfer Learning[J]. Journal of the China Society for Scientific and Technical Information, 2022, 41(4): 412-423.)
|
| [3] |
何晓丽, 牛加明. 三维数字化技术在非物质文化遗产保护中的应用研究——以肇庆端砚为例[J]. 艺术百家, 2016, 32(3): 231-233.
|
| [3] |
(He Xiaoli, Niu Jiaming. Application of 3D Digital Technology in the Intangible Cultural Heritage Protection: Taking Zhaoqing Duan Ink Stone as the Example[J]. Hundred Schools in Arts, 2016, 32(3): 231-233.)
|
| [4] |
赵雪芹, 李天娥, 曾刚. 面向数字人文图像资源的知识元本体构建及关联展示研究[J]. 情报理论与实践, 2022, 45(9): 180-187.
doi: 10.16353/j.cnki.1000-7490.2022.09.024
|
| [4] |
(Zhao Xueqin, Li Tiane, Zeng Gang. Research on Knowledge Meta Ontology Construction and Association Display for Digital Humanities Image Resources[J]. Information Studies: Theory & Application, 2022, 45(9): 180-187.)
doi: 10.16353/j.cnki.1000-7490.2022.09.024
|
| [5] |
许苗苗, 邵波. 我国数字人文发展的脉络、问题及启示[J]. 图书馆学研究, 2020(14): 2-10.
|
| [5] |
(Xu Miaomiao, Shao Bo. The Context, Problems and Enlightenment of Digital Humanities Development in China[J]. Research on Library Science, 2020(14): 2-10.)
|
| [6] |
张红燕, 陈丽平. 基于图书馆领域的国际数字人文研究进展及启示[J]. 图书馆工作与研究, 2022(7): 27-33.
|
| [6] |
(Zhang Hongyan, Chen Liping. Research Progress and Enlightenment of International Digital Humanities Based on Library Field[J]. Library Work and Study, 2022(7): 27-33.)
|
| [7] |
黄水清, 周好, 彭秋茹, 等. 引书的自动识别及文献计量学分析[J]. 情报学报, 2021, 40(12): 1325-1337.
|
| [7] |
(Huang Shuiqing, Zhou Hao, Peng Qiuru, et al. Automatic Recognition and Bibliometric Analysis of Cited Books[J]. Journal of the China Society for Scientific and Technical Information, 2021, 40(12): 1325-1337.)
|
| [8] |
邓君, 王阮. 口述历史档案资源知识图谱与多维知识发现研究[J]. 图书情报工作, 2022, 66(7): 3-16.
|
| [8] |
(Deng Jun, Wang Ruan. Research on Knowledge Map and Multidimensional Knowledge Discovery of Oral History Archives Resources[J]. Library and Information Service, 2022, 66(7): 3-16.)
|
| [9] |
张琪, 王东波, 黄水清, 等. 时间维度下的史籍全文自动重组研究——数字人文视角下的探索[J]. 图书情报知识, 2022, 39(1): 51-60, 147.
|
| [9] |
(Zhang Qi, Wang Dongbo, Huang Shuiqing, et al. Automatic Reorganization of Historical Records from Time Dimension: From the Perspective of Digital Humanities[J]. Documentation, Information & Knowledge, 2022, 39(1): 51-60, 147.)
|
| [10] |
宋雪雁, 钟文敏. 数字人文视角下《谭延闿日记》人物关系挖掘及可视化研究[J]. 情报科学, 2022, 40(6): 25-35.
|
| [10] |
(Song Xueyan, Zhong Wenmin. Character Relationship Mining and Visualization of Tan Yankai’s Diary from the Perspective of Digital Humanities[J]. Information Science, 2022, 40(6): 25-35.)
|
| [11] |
曾子明, 孙守强. 面向敦煌壁画的移动视觉搜索模型研究[J]. 情报资料工作, 2021, 42(2): 104-112.
|
| [11] |
(Zeng Ziming, Sun Shouqiang. Research on Mobile Visual Search Model for Dunhuang Murals[J]. Information and Documentation Services, 2021, 42(2): 104-112.)
|
| [12] |
陈涛, 李惠, 张永娟, 等. LIBRA技术理论及其在史料图像资源中的应用[J]. 大学图书馆学报, 2022, 40(4): 64-74.
|
| [12] |
(Chen Tao, Li Hui, Zhang Yongjuan, et al. LIBRA Technology Theory and Its Application in Historical Image Resources[J]. Journal of Academic Libraries, 2022, 40(4): 64-74.)
|
| [13] |
司徒凌云, 石进, 杨海平, 等. 基于多模态知识图谱的南海疆维权证据链系统构建[J]. 情报杂志, 2021, 40(12): 23-29, 44.
|
| [13] |
(Situ Lingyun, Shi Jin, Yang Haiping, et al. Construction of Evidence Chain System for Rights Protection in the South China Sea Based on Multi-Modal Knowledge Graph[J]. Journal of Intelligence, 2021, 40(12): 23-29, 44.)
|
| [14] |
Oliveira S A, Kaplan F, Lenardo I D. Machine Vision Algorithms on Cadaster Plans[C]// Proceedings of the 12th Annual International Conference of the Alliance of Digital Humanities Organizations. 2017.
|
| [15] |
崔斌, 王东波, 黄水清. 基于典籍文本的农作物时间分布及演化特征研究——以《食货志》为例[J]. 图书情报工作, 2021, 65(14): 90-100.
doi: 10.13266/j.issn.0252-3116.2021.14.011
|
| [15] |
(Cui Bin, Wang Dongbo, Huang Shuiqing. The Analysis of Time Distribution and Evolution Characteristics of Crops in Classics: Taking Shihuozhi as an Example[J]. Library and Information Service, 2021, 65(14): 90-100.)
doi: 10.13266/j.issn.0252-3116.2021.14.011
|
| [16] |
彭秋茹, 范文洁, 黄水清. 数字背景下《春秋经传引得》错漏校订及编纂质量分析[J]. 图书情报工作, 2021, 65(23): 50-57.
doi: 10.13266/j.issn.0252-3116.2021.23.006
|
| [16] |
(Peng Qiuru, Fan Wenjie, Huang Shuiqing. Errors Revision and Compilation Quality Evaluation of Index to Chunqiu Jingzhuan in a Digital Background[J]. Library and Information Service, 2021, 65(23): 50-57.)
doi: 10.13266/j.issn.0252-3116.2021.23.006
|
| [17] |
宋雪雁, 霍晓楠, 刘寅鹏, 等. 数字人文视角下《全唐诗》贬谪诗人的时空轨迹分析[J]. 图书情报工作, 2022, 66(7): 26-34.
doi: 10.13266/j.issn.0252-3116.2022.07.003
|
| [17] |
(Song Xueyan, Huo Xiaonan, Liu Yinpeng, et al. Analysis of the Time-Space Trajectory of Relegated Poets in Complete Collection of Tang Poetry from the Perspective of Digital Humanities[J]. Library and Information Service, 2022, 66(7): 26-34.)
doi: 10.13266/j.issn.0252-3116.2022.07.003
|
| [18] |
陈涛, 张靖, 赵宇翔, 等. 数字人文实践中特藏资源的关联数据实现机制探索——以方志资源为例[J]. 情报理论与实践, 2022, 45(7): 180-187, 147.
doi: 10.16353/j.cnki.1000-7490.2022.07.025
|
| [18] |
(Chen Tao, Zhang Jing, Zhao Yuxiang, et al. Exploration on the Realization Mechanism of Linked Data of Special Collections in Digital Humanities Practice Case Study of Local Chronicles[J]. Information Studies: Theory & Application, 2022, 45(7): 180-187, 147.)
doi: 10.16353/j.cnki.1000-7490.2022.07.025
|
| [19] |
刘政昊, 倪珍妮, 桂文瑄, 等. 知识关联视角下老子“道”学研究多元特征分析与知识体系构建[J]. 现代情报, 2022, 42(7): 42-56, 104.
doi: 10.3969/j.issn.1008-0821.2022.07.004
|
| [19] |
(Liu Zhenghao, Ni Zhenni, Gui Wenxuan, et al. Multi-Characteristic Analysis and Knowledge System Construction of Lao Zi “Taoism” Study from the Perspective of Knowledge Association[J]. Journal of Modern Information, 2022, 42(7): 42-56, 104.)
doi: 10.3969/j.issn.1008-0821.2022.07.004
|
| [20] |
赵雪芹, 李天娥. 面向数字人文的档案领域本体构建研究——以万里茶道档案资料为例[J]. 情报理论与实践, 2022, 45(8): 154-161.
|
| [20] |
(Zhao Xueqin, Li Tiane. Research on Ontology Construction of Archives Field Oriented to Digital Humanities: Take Tea Road Archives as an Example[J]. Information Studies: Theory & Application, 2022, 45(8): 154-161.)
|
| [21] |
张琪, 王东波, 黄水清, 等. 史书多维知识重组与可视化研究——以《史记》为对象[J]. 情报学报, 2022, 41(2): 130-141.
|
| [21] |
(Zhang Qi, Wang Dongbo, Huang Shuiqing, et al. Multi-Dimensional Knowledge Reorganization and Visualization of History Books: Based on Records of the Grand Historian[J]. Journal of the China Society for Scientific and Technical Information, 2022, 41(2): 130-141.)
|
| [22] |
牟加俊, 王建, 何宇清, 等. 一种中国古画印章自动定位算法[J]. 现代电子技术, 2015, 38(2): 96-99 103.
|
| [22] |
(Mou Jiajun, Wang Jian, He Yuqing, et al. Automatic Positioning Algorithm of Ancient Chinese Painting Seal[J]. Modern Electronics Technique, 2015, 38(2): 96-99, 103.)
|
| [23] |
杨琴, 丁莉, 姜鹏, 等. 基于高光谱成像的书画模糊印章信息增强研究[J]. 中国国家博物馆馆刊, 2022(7): 136-147.
|
| [23] |
(Yang Qin, Ding Li, Jiang Peng, et al. Study on Extraction of Blurred Seals in Calligraphy and Painting Based on Hyperspectral Imaging Technology[J]. Journal of National Museum of China, 2022(7): 136-147.)
|
| [24] |
周新光, 褚昊, 吴来明. 最小噪声分离(MNF)变换应用于模糊印章提取的研究[J]. 文物保护与考古科学, 2020, 32(3): 65-69.
|
| [24] |
(Zhou Xinguang, Chu Hao, Wu Laiming. Minimum Noise Fraction (MNF) Transform for Extraction of Blurred Seals[J]. Sciences of Conservation and Archaeology, 2020, 32(3): 65-69.)
|
| [25] |
康雅琪, 孙鹏, 郎宇博, 等. 重构色彩空间下陈旧印章印文自适应Canny检测[J]. 计算机仿真, 2023, 40(8): 230-234, 402.
|
| [25] |
(Kang Yaqi, Sun Peng, Lang Yubo, et al. Adaptive Canny Detection of Obsolete Seals in Reconstructed Color Space[J]. Computer Simulation, 2023, 40(8): 230-234, 402.)
|
| [26] |
葛怀东, 尚弘. 古籍书影中钤印提取技术的探讨[J]. 计算机应用与软件, 2017, 34(5): 189-194.
|
| [26] |
(Ge Huaidong, Shang Hong. Discussion on the Extraction of Seal Prints in Ancient Document Images[J]. Computer Applications and Software, 2017, 34(5): 189-194.)
|
| [27] |
李新利, 邹昌铭, 杨国田, 等. SealGAN: 基于生成式对抗网络的印章消除研究[J]. 自动化学报, 2021, 47(11): 2614-2622.
|
| [27] |
(Li Xinli, Zou Changming, Yang Guotian, et al. SealGAN: Research on the Seal Elimination Based on Generative Adversarial Network[J]. Acta Automatica Sinica, 2021, 47(11): 2614-2622.)
|
| [28] |
杨有, 张汝荟, 许鹏程, 等. 面向民国档案印章分割的改进U-Net[J]. 计算机应用, 2023, 43(3): 943-948.
|
| [28] |
(Yang You, Zhang Ruhui, Xu Pengcheng, et al. Improved U-Net for Seal Segmentation of Republican Archives[J]. Journal of Computer Applications, 2023, 43(3): 943-948.)
|
| [29] |
陈娅娅, 刘全香, 王凯丽, 等. 基于ResNet和迁移学习的古印章文本识别[J]. 计算机工程与应用, 2022, 58(10): 125-131.
doi: 10.3778/j.issn.1002-8331.2101-0247
|
| [29] |
(Chen Yaya, Liu Quanxiang, Wang Kaili, et al. Historical Chinese Seal Text Recognition Based on ResNet and Transfer Learning[J]. Computer Engineering and Applications, 2022, 58(10): 125-131.)
doi: 10.3778/j.issn.1002-8331.2101-0247
|
| [30] |
欧阳欢, 范大昭, 李东子. 多特征融合决策的发票印章识别[J]. 计算机工程与设计, 2018, 39(9): 2842-2847.
|
| [30] |
(Ouyang Huan, Fan Dazhao, Li Dongzi. Invoice Seal Identification Based on Multi-Feature Fusion Decision[J]. Computer Engineering and Design, 2018, 39(9): 2842-2847.)
|
| [31] |
Canny J. A Computational Approach to Edge Detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986, 8(6): 679-698.
pmid: 21869365
|
| [32] |
黄梦涛, 连一鑫. 基于改进Canny算子的锂电池极片表面缺陷检测[J]. 仪器仪表学报, 2021, 42(10): 199-209.
|
| [32] |
(Huang Mengtao, Lian Yixin. Lithium Battery Electrode Plate Surface Defect Detection Based on Improved Canny Operator[J]. Chinese Journal of Scientific Instrument, 2021, 42(10): 199-209.)
|
| [33] |
Nandini D U, Leni E S. Efficient Shadow Detection by Using PSO Segmentation and Region-Based Boundary Detection Technique[J]. The Journal of Supercomputing, 2019, 75(7): 3522-3533.
doi: 10.1007/s11227-018-2292-y
|
| [34] |
司念文, 张文林, 屈丹, 等. 卷积神经网络表征可视化研究综述[J]. 自动化学报, 2022, 48(8): 1890-1920.
|
| [34] |
(Si Nianwen, Zhang Wenlin, Qu Dan, et al. Representation Visualization of Convolutional Neural Networks: A Survey[J]. Acta Automatica Sinica, 2022, 48(8): 1890-1920.)
|
| [35] |
林景栋, 吴欣怡, 柴毅, 等. 卷积神经网络结构优化综述[J]. 自动化学报, 2020, 46(1): 24-37.
|
| [35] |
(Lin Jingdong, Wu Xinyi, Chai Yi, et al. Structure Optimization of Convolutional Neural Networks: A Survey[J]. Acta Automatica Sinica, 2020, 46(1): 24-37.)
|
| [36] |
Krizhevsky A, Sutskever I, Hinton G E. ImageNet Classification with Deep Convolutional Neural Networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
doi: 10.1145/3065386
|
| [37] |
Simonyan K, Andrew Z. Very Deep Convolutional Networks for Large-Scale Image Recognition[C]// Proceedings of the 3rd International Conference on Learning Representations. Washington: ICLR, 2015:1-14.
|
| [38] |
Szegedy C, Liu W, Jia Y Q, et al. Going Deeper with Convolutions[C]// Proceedings of the 28th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 1-9.
|
| [39] |
Huang G, Liu Z, van der Maaten L, et al. Densely Connected Convolutional Networks[C]// Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 2261-2269.
|
| [40] |
张志剑, 刘政昊, 马费成. 面向互联网舆情事件的企业风险识别——基于KGANN模型[J]. 工程管理科技前沿, 2022, 41(1): 65-73.
|
| [40] |
(Zhang Zhijian, Liu Zhenghao, Ma Feicheng. Enterprise Risk Identification for Internet Public Opinion Events——Based on KGANN Model[J]. Frontiers of Science and Technology of Engineering Management, 2022, 41(1): 65-73.)
|
| [41] |
Xu B, Xu Y, Liang J Q, et al. CN-DBpedia: A Never-Ending Chinese Knowledge Extraction System[C]// Proceedings of the 30th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems. Cham: Springer, 2017: 428-438.
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
| |
Shared |
|
|
|
|
| |
Discussed |
|
|
|
|
