MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models
Explore this paper's citation graph
Summary
MiniGPT-4 is presented, which aligns a frozen visual encoder with a frozen advanced LLM, Vicuna, using one projection layer to uncovers that properly aligning the visual features with an advanced large language model can possess numerous advanced multi-modal abilities demonstrated by G PT-4.
- Type
- preprint
- Published
- 2023-04-20
- Cited by
- 3,328
- References
- 62
- Access
- Open access
- OpenAlex
- https://openalex.org/W4366850747
- Semantic Scholar
- https://api.semanticscholar.org/CorpusID:258291930
Keywords
Computer science, Usability, Repetition (rhetorical device), Modal, Code (set theory)
References
- Im2Text: Describing Images Using 1 Million Captioned Photographs
- ReferItGame: Referring to Objects in Photographs of Natural Scenes
- Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
- Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning
- Object Hallucination in Image Captioning
- GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- Connecting Vision and Language with Localized Narratives
- 5分で分かる!? 有名論文ナナメ読み：Jacob Devlin et al. : BERT : Pre-training of Deep Bidirectional Transformers for Language Understanding
- Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts
- Emergent Abilities of Large Language Models
- Zero-Shot Video Question Answering via Frozen Bidirectional Language Models
- GLM: General Language Model Pretraining with Autoregressive Blank Infilling
- Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Training
- Scaling Instruction-Finetuned Language Models
- VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioning
- BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
- A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge
- Language Is Not All You Need: Aligning Perception with Language Models
Cited by
- Language Quantized AutoEncoders: Towards Unsupervised Text-Image Alignment
- Multimodal Chain-of-Thought Reasoning in Language Models
- Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?
- Scaling Vision-Language Models with Sparse Mixture of Experts
- SkinGPT: A Dermatology Diagnostic System with Vision Large Language Model
- mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality
- LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model
- Transfer Visual Prompt Generator across LLMs
- X-LLM: Bootstrapping Advanced Large Language Models by Treating Multi-Modalities as Foreign Languages
- MultiModal-GPT: A Vision and Language Model for Dialogue with Humans
- InternGPT: Solving Vision-Centric Tasks by Interacting with Chatbots Beyond Language
- InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning
- Self-Chained Image-Language Model for Video Localization and Question Answering
- On the Hidden Mystery of OCR in Large Multimodal Models
- Evaluating Object Hallucination in Large Vision-Language Models
- PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering
- Towards A Foundation Model for Generalist Robots: Diverse Skill Learning at Scale via Automated Task and Scene Generation
- X-IQE: eXplainable Image Quality Evaluation for Text-to-Image Generation with Visual Large Language Models
- VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks
- ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities
Related papers
- Enhancing the Human-Machine Interface: An Analysis of Unmanned Aerial Systems (UAS) Using the System Usability Scale (SUS)
- Development of Usability Assessment Indicator of Adjustable Electric Beds for Home Care
- Making energy savings easier: usability metrics for thermostats
- Diagnosis and identification of key issues of usability for reducing medication errors
- Usability assessment of a mobile app for patients with peripherally inserted central catheters
- Usability Evaluation of an Admission, Discharge, and Transfer Information System: A Heuristic Evaluation
- Usability evaluation of ambient assisted living systems using a multi-method approach
- A longitudinal study of usability in health care - does time heal?