SwiftVR: 실시간 원스텝 생성형 비디오 복원 프레임워크
SwiftVR: Real-Time One-Step Generative Video Restoration
- 매체
- Hugging Face
- 게시
- 중요도
- 3/5
- 화제 지수
- 14
한국어 요약
SwiftVR는 라이브 스트림용 실시간 비디오 복원을 위한 생성형 프레임워크다. 마스크 없는 시프트 윈도우 셀프 어텐션과 경량 복원 인식 오토인코더를 통해 고해상도 출력을 소비자급 GPU에서 구현했다. H100에서 2560x1440 31fps, RTX 5090에서 1920x1080 26fps를 달성했다.
원문 발췌 · English
Real-time video restoration (VR) for live streams requires high-resolution outputs under strict per-frame latency constraints. Existing one-step diffusion-based VR models remain difficult to deploy on consumer-grade GPUs due to two main bottlenecks: quadratic spatial attention at high resolutions and the latency-memory overhead of large video autoencoders. We present SwiftVR, a streaming one-step generative VR framework that reduces both bottlenecks under a causal chunk-wise protocol. For attention, mask-free shifted-window self-attention gathers each spatial window into a dense tensor via deterministic indexing, keeping all attention calls on the dense scaled dot-product attention path without masks, cyclic shifts, padding, or hardware-specific sparse kernels. Because SwiftVR uses only standard dense SDPA calls, the trained model transfers to consumer GPUs without retraining or custom kernels. For autoencoding, a lightweight Restoration-aware Autoencoder enables fast chunk-wise decoding while preserving reconstruction quality. On a single H100, SwiftVR sustains 31~FPS at 2560x1440 and 14~FPS at 3840x2160, whereas all compared diffusion-based VR baselines exceed the memory limit at 4K. On a consumer RTX~5090, SwiftVR reaches 26~FPS at 1920x1080. To our knowledge, SwiftVR is the first generative VR model to achieve real-time 1080p streaming on a consumer-grade GPU, while attaining strong no-reference perceptual quality with lower inference cost. Project is available at https://h-oliday.github.io/SwiftVR.