OmniCap-IF benchmarks instruction-following in omni-modal video captioning
OmniCap-IF: Benchmarking and Improving Instruction Following Abilities for Omni-Video Captioning
Hugging FaceTrending 8
Summary
OmniCap-IF is the first benchmark for evaluating instruction-following in omni-modal captioning across visual, audio, and audio-visual modalities with 50 constraint types. The authors release a 54K instruction-tuning dataset and OmniCaptioner-IF model.