Text-to-image models use only word meanings and order from text encoders

Summary

The authors show that text-to-image diffusion transformers rely on only two aspects of text embeddings: merging adjacent tokens into word representations and word order from positional embeddings. A simple bag-of-words representation with position tags achieves visual quality and text fidelity on par with full text embedding-guided generation, revealing that the image model decodes complex linguistic structures rather than the text encoder.