TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding

doi:doi:10.57702/z8j50qno

TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding

TimeChat is a time-sensitive multimodal large language model specifically designed for long video understanding. It incorporates two key architectural contributions: a timestamp-aware frame encoder that binds visual content with the timestamp of each frame, and a sliding video Q-Former that produces a video token sequence of varying lengths to accommodate videos of various durations.

BibTex:

@dataset{Shuhuai_Ren_and_Linli_Yao_and_Shicheng_Li_and_Xu_Sun_and_Lu_Hou_2025,
    abstract = {TimeChat is a time-sensitive multimodal large language model specifically designed for long video understanding. It incorporates two key architectural contributions: a timestamp-aware frame encoder that binds visual content with the timestamp of each frame, and a sliding video Q-Former that produces a video token sequence of varying lengths to accommodate videos of various durations.},
    author = {Shuhuai Ren and Linli Yao and Shicheng Li and Xu Sun and Lu Hou},
    doi = {10.57702/z8j50qno},
    institution = {No Organization},
    keyword = {'long video understanding', 'multimodal learning', 'time-sensitive models'},
    month = {jan},
    publisher = {TIB},
    title = {TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding},
    url = {https://service.tib.eu/ldmservice/dataset/timechat--a-time-sensitive-multimodal-large-language-model-for-long-video-understanding},
    year = {2025}
}