Please wait a minute...
Data Analysis and Knowledge Discovery  2021, Vol. 5 Issue (12): 88-97    DOI: 10.11925/infotech.2096-3467.2021.0643
Current Issue | Archive | Adv Search |
Automatic Detection and Recognition of Oracle Rubbings Based on Mask R-CNN
Liu Fang1,2,3,Li Huabiao1,4(),Ma Jin5,Yan Sheng5,Jin Peiran5
1National Museum of China, Beijing 100006, China
2National Science Library, Chinese Academy of Sciences, Beijing 100190, China
3Department of Library, Information and Archives Management, School of Economics and Management, University of Chinese Academy of Sciences, Beijing 100190, China
4Key Laboratory of Collection Resources Revitalising Technology, Ministry of Culture and Tourism, Beijing 100006, China
5Tianjin Hengda Wenbo S&T Co., Ltd, Tianjin 300384, China
Download: PDF (4916 KB)   HTML ( 12
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] This paper applies the deep learning algorithm to automatically detect and recognize Oracle rubbings, aiming to improve the research and promotion of traditional culture. [Methods] Based on the Mask R-CNN algorithm, we used the three-tuple loss function and rotation angle regression technique to optimize and improve the accuracy of Oracle character classification. [Results] We examined our model with training datasets of Oracle Rubbing Images. The recall of Oracle characters reached 82%, and the detection and identification accuracy reached 95%, which met the expectations of the project. [Limitations] For the severe damaged or ambiguous texts, the performance of our new algorithm needs to be improved. [Conclusions] The proposed model has many practical values and could be further polished.

Key wordsOracle Rubbings      Mask R-CNN      Automatic Detection      Automatic Recognition     
Received: 29 June 2021      Published: 20 January 2022
ZTFLH:  G250  
Fund:Information Development Project of Cultural, Artistic and Tourism Research(MCT2020XZ12)
Corresponding Authors: Li Huabiao,ORCID:0000-0003-4336-0287     E-mail: lihuabiao@chnmuseum.cn

Cite this article:

Liu Fang, Li Huabiao, Ma Jin, Yan Sheng, Jin Peiran. Automatic Detection and Recognition of Oracle Rubbings Based on Mask R-CNN. Data Analysis and Knowledge Discovery, 2021, 5(12): 88-97.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2021.0643     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y2021/V5/I12/88

Overall Technical Route
Data Acquisition and Labeling
分类 学者 算法类型 特点
传统方法 刘永革等[21] SVM 采用分块直方图的方式提取甲骨文的特征形态
周新伦等[22] 迪卡尔直角坐标 抽取甲骨文的端点、差点、块、孔作为拓扑特征
吕肖庆等[23] 曲率直方图 结合整体统计结果和局部曲率计算甲骨文字的特征向量
基于深度学习的方法 鲁绪正等[24] Capsule 将甲骨文同时拆解成多个构件进行识别
邢济慈[25] YOLOv3 文字自动定位,单字自动检测识别
徐贵良[26] YOLOv2 甲骨文构件自动检测识别
Oracle Identification Study Case
The Main Algorithm Architecture of Oracle Rubbings Detection and Recognition
均值 检测性能 识别性能
R P F1 R P F1
字按图
平均
0.913 9 0.971 8 0.939 4 0.902 7 0.958 6 0.927 4
字累加
平均
0.898 9 0.982 8 0.939 0 0.882 0 0.964 3 0.921 3
Statistics of Oracle Rubbings Retrieval and Recognition
算法 检测性能 识别性能
R P F1 R P F1
Faster R-CNN 0.370 7 0.819 3 0.510 5 0.167 5 0.369 7 0.230 5
本文算法 0.785 3 0.948 4 0.854 4 0.566 6 0.668 3 0.610 0
Performance Comparison Analysis with Faster R-CNN
训练样本 阶段训练 本次训练
训练数据集 人工标注拓片数量 500张 3 300张
甲骨文字符类别 499个 831个
甲骨文字符个数 约8 000个 约40 000个
样本分布 极度不均衡,个别类别只有一个样本 不均衡,个别类别样本较少
数据增广方式 随机缩放、旋转、透视、亮度、剪切变换;拓片拟真
训练参数 训练回合数 45 887
每回合训练次数 4 000 2 000
每次训练样本数 4 1
训练结果 甲骨文字符检测准确率 77% 98%
甲骨文字符识别准确率 64% 96%
Comparative Analysis of the Two Model Training Data
Oracle Tablets Retrieval and Identification Test Case
[1] 中国国家博物馆. 中国国家博物馆馆藏文物研究丛书: 甲骨卷[M]. 上海: 上海古籍出版社, 2007: 292-293.
[1] (National Museum of China. Studies of the Collections of the National Museum of China: Oracle [M]. Shanghai: Shanghai Ancient Books Publishing House, 2007: 292-293.)
[2] 中国国家博物馆, 中国书法家协会. 中国国家博物馆典藏甲骨文金文集粹[M]. 合肥: 安徽美术出版社, 2015: 5-8.
[2] (National Museum of China, Chinese Calligraphers Association. A Collection of Oracle and Bronze of National Museum of China[M]. Hefei: Anhui Fine Arts Publishing House, 2015: 5-8.)
[3] “绝学”不绝于耳——从近十年国家社科基金选题看甲骨文研究进展[EB/OL]. [2021-04-09]. https://www.sohu.com/a/424326096_488440.
[3] (Lost Knowledge is Endless——Viewing the Research Progress of Oracle Inscriptions from the Topic Selection of the National Social Science Fund[EB/OL].[2021-04-09]. https://www.sohu.com/a/424326096_488440.)
[4] 2018年国家社会基金特别委托项目成果举要[EB/OL].[2021-04-09]. http://www.nopss.gov.cn/n1/2019/0703/c219507-31211146.html.
[4] (The Results of Special Commissioned Projects of the National Social Fund in 2018 [EB/OL]. [2021-04-09]. http://www.nopss.gov.cn/n1/2019/0703/c219507-31211146.html.)
[5] 熊晶, 韩胜伟. 甲骨文研究中跨模态知识图谱的重要性刍议[J]. 殷都学刊, 2020, 41(3): 60-64,97.
[5] (Xiong Jing, Han Shengwei. On the Importance of Cross-modal Knowledge Graph in Oracle Inscriptions Research[J]. Yindu Academic Journal, 2020, 41(3): 60-64, 97.)
[6] 甲骨文信息处理重点实验室[EB/OL]. [2021-04-09]. http://jgwsys.aynu.edu.cn/index.htm.
[6] (Key Laboratory of Oracle Information Processing[EB/OL].[2021-04-09]. http://jgwsys.aynu.edu.cn/index.htm.)
[7] 甲骨文拓片资料库[EB/OL].[2021-04-09]. https://ndweb.iis.sinica.edu.tw/rub_public/System/Bone/home2.htm.
[7] (Oracle Rubbing Database [EB/OL].[2021-04-09]. https://ndweb.iis.sinica.edu.tw/rub_public/System/Bone/home2.htm.)
[8] Cuneiform Digital Library Initiative[EB/OL]. [2021-04-09]. https://cdli.ucla.edu/.
[9] Database of Neo-Sumerian Texts[EB/OL]. [2021-04-09]. http://bdtns.filol.csic.es/.
[10] Electronic Pennsylvania Sumerian Dictionary[EB/OL]. [2021-04-09]. http://psd.museum.upenn.edu/nepsd-frame.html.
[11] Bogacz B, Gertz M, Mara H. Cuneiform Character Similarity Using Graph Representations [C]//Proceedings of the 20th Computer Vision Winter Workshop. 2015: 77-83.
[12] Dencker T, Klinkisch P, Maul S M, et al. Deep Learning of Cuneiform Sign Detection with Weak Supervision Using Transliteration Alignment[J]. PLoS One, 2020, 15(12): e0243039.
doi: 10.1371/journal.pone.0243039
[13] He K M, Gkioxari G, Dollár P, et al. Mask R-CNN[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2): 386-397.
doi: 10.1109/TPAMI.34
[14] ResNet-50[EB/OL].[2021-07-02]. https://blog.csdn.net/Cheungleilei/article/details/103610799.
[15] Triplet Loss[EB/OL].[2021-07-02]. https://machinelearning.wtf/terms/triplet-loss/.
[16] 郭沫若. 甲骨文合集[M]. 北京: 中华书局, 1982.
[16] (Guo Moruo. Oracle Collection[M]. Beijing: China Book Bureau, 1982.)
[17] 李宗焜. 甲骨文字编[M]. 北京: 中华书局, 2012.
[17] (Li Zongkun. Oracle’s Words Made Up[M]. Beijing: China Book Bureau, 2012.)
[18] 刘钊, 冯克坚. 甲骨文常用字字典[M]. 北京: 中华书局, 2019.
[18] (Liu Zhao, Feng Kejian. Oracle’s Common Word Dictionary[M]. Beijing: China Book Bureau, 2019.)
[19] Gupta A, Vedaldi A, Zisserman A. Synthetic Data for Text Localisation in Natural Images [C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016: 2315-2324.
[20] 张颐康, 张恒, 刘永革, 等. 基于跨模态深度度量学习的甲骨文字识别[J]. 自动化学报, 2021, 47(4): 791-800.
[20] (Zhang Yikang, Zhang Heng, Liu Yongge, et al. Oracle Character Recognition Based on Cross-Modal Deep Metric Learning[J]. Acta Automatica Sinica, 2021, 47(4): 791-800.)
[21] 刘永革, 刘国英. 基于SVM的甲骨文字识别[J]. 安阳师范学院学报, 2017(2): 54-56.
[21] (Liu Yongge, Liu Guoying. Oracle Bone Inscription Recognition Based on SVM[J]. Journal of Anyang Normal University, 2017(2): 54-56.)
[22] 周新伦, 李锋, 华星城, 等. 甲骨文计算机识别方法研究[J]. 复旦学报(自然科学版), 1996, 35(5): 481-486.
[22] (Zhou Xinlun, Li Feng, Hua Xingcheng, et al. A Method of Jia Gu Wen Recognition Based on a Two-Level Classification[J]. Journal of Fudan University (Natural Science), 1996, 35(5): 481-486.)
[23] 吕肖庆, 李沫楠, 蔡凯伟, 等. 一种基于图形识别的甲骨文分类方法[J]. 北京信息科技大学学报(自然科学版), 2010, 25(S2): 92-96.
[23] (Lv Xiaoqing, Li Monan, Cai Kaiwei, et al. A Graphic-Based Method for Chinese Oracle-Bone Classification[J]. Journal of Beijing Information Science & Technology University, 2010, 25(S2): 92-96.)
[24] 鲁绪正, 蔡恒进, 林莉. 基于Capsule网络的甲骨文构件识别方法[J]. 智能系统学报, 2020, 15(2): 243-254.
[24] (Lu Xuzheng, Cai Hengjin, Lin Li. Recognition of Oracle Radical Based on the Capsule Network[J]. CAAI Transactions on Intelligent Systems, 2020, 15(2): 243-254.)
[25] 邢济慈. 基于深度卷积神经网络的甲骨文字检测技术研究[D]. 郑州: 郑州大学, 2020.
[25] (Xing Jici. Research of Oracle Bone Inscription Detection Based on Deep Convolutional Neural Network[D]. Zhengzhou: Zhengzhou University, 2020.)
[26] 徐贵良. 基于语义分析的深度学习的甲骨文部首检测的研究[D]. 南昌: 江西科技师范大学, 2020.
[26] (Xu Guiliang. Research on Oracle Bone Radical Detection Based in Deep Learning of Semantic Analysis[D]. Nanchang: Jiangxi Science and Technology Normal University, 2020.)
[27] Softmax Regression [EB/OL]. [2021-07-02]. https://www.biaodianfu.com/softmax-regression.html.
[28] Python[EB/OL]. [2021-07-02]. https://www.python.org/.
[29] TensorFlow[EB/OL]. [2021-07-02]. https://www.tensorflow.org/.
[30] Faster R-CNN[EB/OL].[2021-07-02]. https://paperswithcode.com/method/faster-r-cnn.
[31] Kipf T. Graph Convolutional Network[EB/OL]. [2021-07-02]. https://tkipf.github.io/graph-convolutional-networks/.
[32] Using RANSAC for Estimating Geometric Transforms in Computer Vision [EB/OL].[2021-07-02]. https://www.mathworks.com/discovery/ransac.html.
[33] 江一苇, 顾幸生. 基于网格加速与顺序选取策略的图像匹配算法[J]. 华东理工大学学报(自然科学版),DOI: 10.14135/j.cnki.1006-3080.20210401002.
doi: 10.14135/j.cnki.1006-3080.20210401002
[33] (Jiang Yiwei, Gu Xingsheng. Image Matching Algorithm Based on Grid Acceleration and Sequential Selection Strategy[J]. Journal of East China University of Science and Technology, DOI: 10.14135/j.cnki.1006-3080.20210401002.)
doi: 10.14135/j.cnki.1006-3080.20210401002
[1] Wang Xiaomei,Deng Qiping. Auto-Identifying Research Area Groups in Science Map[J]. 现代图书情报技术, 2016, 32(4): 48-55.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn