95 datasets found

The WavCaps dataset contains chatGPT-assisted weakly-labeled audio captioning data.

The Multimodal Visual Patterns (MMVP) benchmark is a dataset used to evaluate the visual question answering capabilities of multimodal large language models (MLLMs).

Degree datasets are constructed by gradually adjusting the degree of alignment between image and text.

The dataset used in the paper is a multimodal learning task for robots.

A benchmark for evaluating the perception ability of Large Vision-Language Models (LVLMs) via various subtasks and scenarios.

Dysca is a dynamic and scalable benchmark for evaluating the perception ability of Large Vision-Language Models (LVLMs) via various subtasks and scenarios.

Multimodal C4 (mmc4) is a public, billion-scale corpus of images and text, constructed from public webpages contained in the cleaned English c4 corpus.

TCGA-OMICS: A comprehensive dataset of genomic, transcriptomic, and proteomic data from The Cancer Genome Atlas Program

MUGEN-GAME: A large-scale and multimodal dataset for video-audio-text multimodal understanding and generation

Training transitive and commutative multimodal transformers with LoReTTa

A dialogue dataset for open-domain conversation models.

A dialogue dataset for personalizing dialogue agents.

A dialogue dataset with photo sharing behavior for joint image-text modeling.

A multi-modal dialogue dataset created by replacing text with semantically relevant images.

A large-scale multi-modal dialogue dataset created by leveraging the automatic pipeline with filtering using CLIP similarity.

Video question answering (VideoQA) requires systems to understand the visual information and infer an answer for a natural language question from it.

VideoIC dataset for automatic live video commenting

Livebot dataset for automatic live video commenting

Sentiment-oriented Transformer-based Variational Autoencoder (So-TVAE) for Live Video Commenting

InternVid: A large-scale video-text dataset for multimodal understanding and generation.