AI Daybook
AI Daybook
Today's issue
PaperResearch

WorldDraft: evaluating video models as generative renderers for 4D worlds

Original titleWorldDraft: If Code Builds the World, Can Video Models Just Render It?

Outlet
Hugging Face
Published
Importance
4/5
Trending
13

Summary

The authors introduce WorldDraft, a benchmark of 432 paired test cases across 48 executable 4D world families, to evaluate whether video models can reliably render 4D worlds specified in code. Evaluations of 12 models reveal operational tensions between world preservation and visual realization, and source leakage of up to 64%. The authors propose WorldDraft-Primer, a visual prompting strategy that improves both objectives on frozen video renderers.

Original excerpt

Multimodal coding agents decouple 4D world construction from visual generation, repurposing video models from world generators toward generative renderers. This division of labor raises a fundamental question: if code builds the world, can video models just render it? A reliable generative renderer hinges on two complementary criteria: (i) world preservation, retaining prescribed 4D kinematics and camera paths; and (ii) visual realization, persistently binding text-specified target attributes over time. To evaluate this capability, we introduce WorldDraft, a benchmark of 432 strictly paired test cases across 48 executable 4D world families spanning six dynamic challenge categories. Under a 3 times 3 factorial design, WorldDraft orthogonally crosses draft granularity (abstract proxy to previs) and appearance alignment (neutral to conflicting) under identical dynamics. Extensive evaluations across 12 representative models reveal that: (I) world preservation and visual realization remain in stark operational tension; (II) richer draft scaffolding anchors structure but paradoxically impairs appearance fidelity; and (III) conflicting source cues trigger severe source leakage (up to 64%), exposing fragile cross-modal disentanglement. To mitigate these bottlenecks, we propose WorldDraft-Primer, a plug-and-play visual prompting strategy that consistently improves both objectives across frozen video renderers. Together, WorldDraft and WorldDraft-Primer lay a foundation for bridging programmatic 4D simulation with video generation, advancing physically grounded generative rendering.

#multimodal#benchmark#world model#video generation
Read the original
See all →