광학 추론: 이미지를 텍스트 이상의 추론 매체로 재정의
Optical Reasoning: Rethinking Images as an Expressive Reasoning Medium Beyond Text
- 매체
- Hugging Face
- 게시
- 중요도
- 3/5
- 화제 지수
- 5
한국어 요약
이미지를 텍스트 대신 추론의 주요 매체로 사용하는 '광학 추론(Optical Reasoning)' 기법이 제안됐다. 타이포그래피 기반과 그래픽 기반 두 가지 방식으로 구현되며, 수학·과학·멀티모달 추론 벤치마크에서 텍스트 추론과 동등하거나 우수한 성능을 보이면서 토큰 효율성은 1.96배 향상시켰다.
원문 발췌 · English
Chain-of-Thought (CoT) improves the performance of Large Language Models (LLMs) and has been extended to Multimodal Large Language Models (MLLMs). More recent work further moves from text-based multimodal reasoning toward interleaved-modal reasoning, where intermediate steps can incorporate both textual rationales and visual evidence. In this work, we propose a bolder and more ambitious idea: could images alone serve as the reasoning medium for both language and multimodal tasks? To explore this, we propose optical reasoning, which treats images as a standalone reasoning medium. We instantiate this concept with two variants: typographic-based optical reasoning, which optimizes visual layouts for compact rationale rendering, and graphical-based optical reasoning, which composes text and graphical elements into structured visual rationales. Across mathematical, scientific, and interleaved-modal reasoning benchmarks, optical reasoning can match or even exceed traditional text reasoning while reducing reasoning tokens by an average of 28.57% on language tasks and 16% on multimodal tasks, achieving 1.96 times the token efficiency of text reasoning. These results show that images can effectively and efficiently encode rationales while providing a unified visual canvas for reasoning.