Clartemis: Clip-Based Attention-Based Retrieval with Text-Explicit Matching and Implicit Similarity - Tran, Hoang-Anh; Le, Hoanh-Su; Nguyen, Phuc (International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP) (Online), 2024)
This study tackles the challenge of image retrieval using text feedback, a crucial task in e-commerce. To address this, we introduce an improved framework called CLARTEMIS, built upon the foundation of ARTEMIS [4]. Our proposed approach incorporates a pre-trained vision-language model to align image and text features within a unified semantic space. This integration not only streamlines multimodal representation learning but also enhances the consistency of the feature space for correlation modeling. The CLARTEMIS framework utilizes a robust joint embedding strategy, effectively aligning reference images, modification texts, and target images. Comprehensive experiments on a benchmark dataset validate the effectiveness of our method, demonstrating notable improvements in retrieval accuracy and underscoring the potential of pre-trained models in advancing multimodal retrieval systems.