Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal Retrieval
Explore this paper's citation graph
Summary
A novel fine-tuning framework that turns any pretrained text-image multi-modal model into an efficient retrieval model, based on a cooperative retrieve-and-rerank approach that combines a twin networks and a cross-encoder component for a more nuanced ranking of the retrieved small set of items.
- Type
- article
- Published
- 2021-03-22
- Cited by
- 65
- References
- 82
- Access
- Open access
- OpenAlex
- https://openalex.org/W3139017368
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:232307891
Keywords
Computer science, Encoder, Modal, Scalability, Information retrieval
References
- Learning Two-Branch Neural Networks for Image-Text Matching Tasks
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Answering English questions by computer: a survey
- Near-Optimal Hashing Algorithms for Approximate Nearest Neighbor in High Dimensions
- Efficient search for approximate nearest neighbor in high dimensional spaces
- Approximate nearest neighbor queries in fixed dimensions
- Im2Text: Describing Images Using 1 Million Captioned Photographs
- DeViSE: A Deep Visual-Semantic Embedding Model
- An Investigation of Practical Approximate Nearest Neighbor Algorithms
- Multi30K: Multilingual English-German Image Descriptions
- An optimal algorithm for approximate nearest neighbor searching fixed dimensions
- Dual Attention Networks for Multimodal Reasoning and Matching
- Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models
- Billion-Scale Similarity Search with GPUs
- In Defense of the Triplet Loss for Person Re-Identification
- Proceedings of the 2001 Conference on Empirical Methods in Natural Language Processing
- Image Pivoting for Learning Multilingual Multimodal Representations
- Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering
- Findings of the Second Shared Task on Multimodal Machine Translation and Multilingual Image Description
- Dual-path Convolutional Image-Text Embeddings with Instance Loss
Cited by
- Thinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformers
- xGQA: Cross-Lingual Visual Question Answering
- Assorted Attention Network for Cross-Lingual Language-to-Vision Retrieval
- MixBERT for Image-Ad Relevance Scoring in Advertising
- Multi-modal Dictionary BERT for Cross-modal Video Search in Baidu Advertising
- Inflate and Shrink:Enriching and Reducing Interactions for Fast Text-Image Retrieval
- LoopITR: Combining Dual and Cross Encoder Architectures for Image-Text Retrieval
- Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval
- CADG: A Model Based on Cross Attention for Domain Generalization
- Subverting Fair Image Search with Generative Adversarial Perturbations
- Efficient Image and Sentence Matching
- Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone
- Cross-Probe BERT for Fast Cross-Modal Search
- Parameter-Efficient Prompt Tuning Makes Generalized and Calibrated Neural Text Retrievers
- FashionViL: Fashion-Focused Vision-and-Language Representation Learning
- CrispSearch: low-latency on-device language-based image retrieval
- Decoupled Cross-Modal Phrase-Attention Network for Image-Sentence Matching
- U-BERT for Fast and Scalable Text-Image Retrieval
- Learning Explicit and Implicit Dual Common Subspaces for Audio-visual Cross-modal Retrieval
- Improving Visual-Semantic Embedding with Adaptive Pooling and Optimization Objective
Related papers
- A Dynamic Scalable Asynchronous Message Model Based on Distributed Objects
- Distributed memory parallel approaches for HEVC encoder
- Scalable Database Management in Cloud Computing
- Performance Analysis of Controlled Scalability in Unstructured Peer-to-Peer Networks
- A Kind of P2P Network For Collaboration Systems
- Desynchronization of simulation and optimization algorithms in HPC Environment
- A survey of mapreduce based parallel processing technologies
- ASCruiser:A P2P Monitoring System for Live Media Streaming
- Design and implementation of recommendation system for E-commerce on Hadoop