[Objective] Summarize the fundamental strategies and core issues in Cross-Modal Information Retrieval (CMIR) based on correlation, and do research about the pros and cons of using partial least squares in feature subspace projection in order to improve retrieval effect. [Methods] Based on Wikipedia CMIR dataset, LDA and BOW models are used as a characteristic expression of text and image resources, cosine distance as the similarity measure, and the least squares method is used to learn subspace projection function replacing canonical correlation analysis method. [Results] Using comparative analysis of the influence of three features subspace projection methods named canonical correlation analysis, partial least squares regression, partial least squares correlation on CMIR results according to three retrieval evaluation indicators that are P@K, MAP and NDCG, and the results show that partial least squares correlation obtains the best results. [Limitations]In dealing with data, partial least squares method assumes a linear relationship between the data and an orthogonal relationship between the data base vectors, therefore the non-linear, non-orthogonal problem can not be solved. [Conclusions] Feature subspace projection learning by using partial least squares correlation is more consistent with original spatial information, and CMIR results are more stable.
丁恒, 陆伟. 基于相关性的跨模态信息检索研究*[J]. 现代图书情报技术, 2016, 32(1): 17-23.
Heng Ding, Wei Lu. A Study on Correlation-based Cross-Modal Information Retrieval. New Technology of Library and Information Service, DOI：10.11925/infotech.1003-3513.2016.01.04.
(Duan Ruixue, Wang Xiaojie, Sun Yueping, et al.Clustering User Goals Based on Hierarchical Dirichlet Process Topic Model[J]. Journal of Beijing University of Posts and Telecommunications, 2011, 34(S1): 55-58.)
Wu F, Zhang H, Zhuang Y.Learning Semantic Correlations for Cross-Media Retrieval [C]. In: Proceedings of IEEE International Conference on Image Processing, Atlanta, USA. IEEE, 2006: 1465-1468.
(Ming Junren, He Chao.Research on Cross-media Retrieval Method in Digital Library Based on Semantic Association Mining[J]. Library and Information Service, 2013, 57(7): 101-105.)
张鸿. 基于相关性挖掘的跨媒体检索研究[D]. 杭州: 浙江大学, 2007.
(Zhang Hong.Correlation Mining Based Cross- media Retrieval [D]. Hangzhou: Zhejiang University, 2007.)
Rasiwasia N, Costa Pereira J, Coviello E, et al.A New Approach to Cross-modal Multimedia Retrieval [C]. In: Proceedings of the International Conference on Multimedia. ACM, 2010: 251-260.
Costa Pereira J, Coviello E, Doyle G, et al.On the Role of Correlation and Abstraction in Cross-modal Multimedia Retrieval[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(3): 521-535.
(Liu Yang, Zheng Fengbin, Jiang Baoqing, et al.Research of Cross-media Information Retrieval Model Based on Multimodal fusion and Temporal-spatial Context Semantic[J]. Journal of Computer Applications, 2009, 29(4): 1182-1187.)
Zhai X, Peng Y, Xiao J.Cross-modality Correlation Propagation for Cross-media Retrieval [C]. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Kyoto, Japan. IEEE, 2012: 2337-2340.