1School of Mathematics and Information Science, Nanjing Normal University of Special Education, Nanjing 210038, China 2Braille and Sign Language Research Center, Nanjing Normal University of Special Education, Nanjing 210038, China 3Jiangsu Provincial Key Laboratory of Data Engineering and Knowledge Service, Nanjing 210023, China
[Objective] This paper extracts and organizes knowledge from multimodal sign language resources and constructs a corpus for related research. It meets the public’s urgent demands to obtain sign language knowledge. [Context] The new multimodal corpus is suitable for mining sign language knowledge, which addresses low information levels, disordered resource organization, and difficult utilization of sign language knowledge. [Methods] Firstly, we constructed the multi-modal feature annotation system for sign language vocabulary. Secondly, we formulated the feature coding scheme of the vocabulary and implemented multi-level annotation. Finally, we established the graph model for sign language vocabulary and the Neo4j database to store and visualize. [Results] The vocabulary data are from the national sign language vocabulary corpus. Over 10 000 sign language vocabulary multimodal annotation has been completed, and we realized the whole process of constructing a multimodal corpus. [Conclusions] The new corpus increases knowledge retrieval of hand shape, movement, expression, and posture, which greatly improves the usability of the sign language corpus.
张艳琼, 朱兆松, 赵晓驰. 面向手语语言学的中国手语词汇多模态语料库构建研究*[J]. 数据分析与知识发现, 2023, 7(10): 144-155.
Zhang Yanqiong, Zhu Zhaosong, Zhao Xiaochi. Constructing Multimodal Corpus of Chinese Vocabulary for Sign Language Linguistics. Data Analysis and Knowledge Discovery, 2023, 7(10): 144-155.
(Xiang Anling, Gao Shuang, Peng Yingtong, et al. Knowledge Reorganization and Scene Reconstruction: A Metaverse for Digital Resources Management[J]. Documentation, Information & Knowledge, 2022, 39(1): 30-38.)
(Zhang Haoyu, Wang Tianbao, Li Mengze, et al. Comprehensive Review of Visual-Language-Oriented Multimodal Pre-Training Methods[J]. Journal of Image and Graphics, 2022, 27(9): 2652-2682.)
[3]
姚登峰. 手语计算概论[M]. 北京: 科学出版社, 2022.
[3]
(Yao Dengfeng. A Guide to Sign Language Computing[M]. Beijing: Science Press, 2022.)
(Qiu Yunfeng, Yao Dengfeng, Li Rong, et al. Introduction to Chinese Sign Language Linguistics[M]. Beijing: China International Broadcasting Press, 2018.)
(China Disabled Persons’ Federation. Second Phase National Action Plan for Standardization of Sign Language and Braille (2021-2025)[EB/OL](2021-11-29)[2022-11-26]. https://www.cdpf.org.cn//zwgk/zcwj/wjfb/fe1a8761eb2d40bc9467179bdac0b551.htm.)
(Zhao Xiaochi, Ren Yuanyuan, Ding Yong. On the Construction and Application of China’s Sign Language Vocabulary Corpus[J]. Chinese Journal of Special Education, 2017(1): 43-47.)
[7]
张帜. Neo4j权威指南[M]. 北京: 清华大学出版社, 2017.
[7]
(Zhang Zhi. Neo4j Authoritative Guide[M]. Beijing: Tsinghua University Press, 2017.)
[8]
Lucas C, Bayley R. Variation in ASL: The Role of Grammatical Function[J]. Sign Language Studies, 2005, 6(1): 38-75.
doi: 10.1353/sls.2006.0005
[9]
Johnston T A, Schembri A. Australian Sign Language (Auslan): An Introduction to Sign Language Linguistics[M]. Cambridge, UK: Cambridge University Press, 2007.
Caselli N K, Sehyr Z S, Cohen-Goldberg A M, et al. ASL-LEX: A Lexical Database of American Sign Language[J]. Behavior Research Methods, 2017, 49(2): 784-801.
doi: 10.3758/s13428-016-0742-0
pmid: 27193158
[12]
Sehyr Z S, Caselli N, Cohen-Goldberg A M, et al. The ASL-LEX 2.0 Project: A Database of Lexical and Phonological Properties for 2,723 Signs in American Sign Language[J]. The Journal of Deaf Studies and Deaf Education, 2021, 26(2): 263-277.
doi: 10.1093/deafed/enaa038
[13]
Fenlon J, Cormier K, Rentelis R, et al.BSL Signbank: A Lexical Database of British Sign Language[DB/OL]. [2022-11-26]. http://bslsignbank.ucl.ac.uk.
[14]
Schembri A, Fenlon J, Rentelis R, et al. British Sign Language Corpus Project: A Corpus of Digital Video Data and Annotations of British Sign Language[DB/OL]. [2022-11-26]. http://www.bslcorpusproject.org.
[15]
Fenlon J, Cormier K, Schembri A. Building BSL SignBank: The Lemma Dilemma Revisited[J]. International Journal of Lexicography, 2015, 28(2): 169-206.
doi: 10.1093/ijl/ecv008
(National Office for Philosophy and Social Sciences. Sign Language Corpus Research Based on Chinese and Some Minority Languages[R/OL]. [2022-11-26]. http://www.nopss.gov.cn/GB/352519/355466/.)
(Wu Ruizhu, Li Hanjing, Lv Huihua, et al. Construction of Parallel Corpus of Chinese and Sign Language for ELAN[J]. Journal of Chinese Information Processing, 2019, 33(2): 43-50.)
[20]
黄晓晓. 基于情景语料库的自然手语构词研究[D]. 南京: 南京师范大学, 2012.
[20]
(Huang Xiaoxiao. Study of Natural Sign Language Word Formation Based on Situational Corpus[D]. Nanjing: Nanjing Normal University, 2012.)
[21]
周闯. 基于中文分词的聋校小学记事文手语语料库构建研究[D]. 武汉: 华中师范大学, 2019.
[21]
(Zhou Chuang. Research on the Construction of Deaf Primary School Text Corpus Based on Chinese Word Segmentation Technology[D]. Wuhan: Central China Normal University, 2019.)
[22]
Stokoe W C. Sign Language Structure[M]. Buffalo: University of Buffalo Press, 1960.
[23]
Liddell S K, Johnson R E. American Sign Language: The Phonological Base[J]. Sign Language Studies, 1989, 64(1): 195-277.
[24]
Sandler W. Phonological Representation of the Sign: Linearity and Nonlinearity in American Sign Language[M]. Dordrecht, Holland: Foris Publications, 1989.
[25]
Brentari D. A Prosodic Model of Sign Language Phonology[M]. Cambridge, Mass: MIT Press, 1998.
[26]
Tang G. Hong Kong Sign Language: A Trilingual Dictionary with Linguistic Descriptions[M].The Chinese University Press, 2007.
[27]
Battison R M, Baird E. Lexical Borrowing in American Sign Language[OL]. [2022-01-01] https://api.semanticscholar.org/CorpusID:60545823.
(Zhang Jisheng, Wu Yanhong. The Underlying Handshapes and Their Feature Specification of Shanghai Sign Language[J]. Contemporary Linguistics, 2018, 20(4): 572-586.)
[29]
骆维维. 《中国手语》手形研究[D]. 北京: 北京师范大学, 2008.
[29]
(Luo Weiwei. Study on the Handshape of Chinese Sign Language[D]. Beijing: Beijing Normal University, 2008.)
[30]
衣玉敏. 上海手语的语音调查报告[D]. 上海: 复旦大学, 2008.
[30]
(Yi Yumin. The Survey of the Phonology of Shanghai Sign Language[D]. Shanghai: Fudan University, 2008.)
[31]
ELAN (Version 6.2)[DB/OL]. [2022-01-24]. https://archive.mpi.nl/tla/elan.
（编号DEZ101）示例