Please wait a minute...
Data Analysis and Knowledge Discovery
Current Issue | Archive | Adv Search |
Deep Cross-modal Hashing Based on Intra-modal Similarity and Semantic Preservation
Li Tianyu,Liu Libo
(School of Information Engineering, Ningxia University, Ningxia Hui Autonomous Region, Yinchuan 750021,China)
Download:
Export: BibTeX | EndNote (RIS)      
Abstract  

[Objective] In order to solve the problem that most of existing cross-modal hashing methods only consider inter-modal similarity, and cannot make full use of the label semantic information, thus ignoring the heterogeneous data details and leading to the loss of semantic information.

[Methods] Firstly, the methods uses Euclidean distance and Tanimoto coefficient to measure the intra-modal similarity of data from images and texts respectively; Then the weighted values of the two are used to measure the inter-modal similarity to make full use of the detailed information of heterogeneous data; After that, the discriminativeness of the hash code is improved by preserving the semantic information of the data label, and the loss of semantic information is prevented; Finally, the quantization loss is calculated on the generated hash code and the hash bit balance constraint is applied to further improve the quality of the hash code.

[Results] Compare with 11 methods,the highest mAP for the retrieval tasks of image to text and text to image increase by 9.5% and 5.8%, respectively, on MIRFlickr25k and 4.7% and 1.1% on NUS-WIDE.

[Limitations] Model training relies on label information, and the effect decreases in unsupervised and semi-supervised situations.

[Conclusions] The proposed method can retain the detailed information of heterogeneous data and prevent the loss of semantic information, which effectively improves the model retrieval performance.


Key words Cross-modal retrieval      Cross-modal hashing      Intra-modal similarity preservation      Semantic preservation      
Published: 29 July 2022
ZTFLH:  TP391.4  

Cite this article:

Li Tianyu, Liu Libo. Deep Cross-modal Hashing Based on Intra-modal Similarity and Semantic Preservation . Data Analysis and Knowledge Discovery, 0, (): 1-.

URL:

https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/10.11925/infotech.2096-3467.2022-0536     OR     https://manu44.magtech.com.cn/Jwk_infotech_wk3/EN/Y0/V/I/1

[1] Li Tianyu, Liu Libo. Deep Cross-modal Hashing Based on Intra-modal Similarity and Semantic Preservation[J]. 数据分析与知识发现, 2023, 7(5): 105-115.
[2] Zhu Lu, Deng Fang, Liu Kun, He Tingting, Liu Yuanyuan. Cross-Modal Retrieval Based on Semantic Auto-Encoder and Hash Learning[J]. 数据分析与知识发现, 2021, 5(12): 110-122.
[3] Zhu Lu,Tian Xiaomeng,Cao Sainan,Liu Yuanyuan. Subspace Cross-modal Retrieval Based on High-Order Semantic Correlation[J]. 数据分析与知识发现, 2020, 4(5): 84-91.
  Copyright © 2016 Data Analysis and Knowledge Discovery   Tel/Fax:(010)82626611-6626,82624938   E-mail:jishu@mail.las.ac.cn