IQuest-Q1, 에이전트 CLI 시스템을 위한 기초 모델 공개
IQuest-Q1: Advancing Foundation Capabilities for Agentic CLI Systems
- 매체
- Hugging Face
- 게시
- 중요도
- 2/5
- 화제 지수
- 20
한국어 요약
320B 파라미터의 희소 MoE 기반 에이전트 파운데이션 모델 IQuest-Q1이 공개됐다. 코딩과 일반 작업 능력을 갖추며 오픈소스로 제공된다.
원문 발췌 · English
We present IQuest-Q1, a sparse Mixture-of-Experts (MoE) agentic foundation model with 320B total parameters (15B activated). IQuest-Q1 is developed through a three-stage process: (1) stable and efficient pre-training, combining Stable SplitNorm, Muon, and muP with a two-stage curriculum whose second stage increases the coding-data mixture, preserving a wide stability margin between safe and optimal learning rates; (2) fine-grained mid-training, expanding the data mixture across domains and formats and progressively extending the context; and (3) multi-scaffold with multi-version post-training, combining SFT, domain-specialized RL, and MOPD across various scaffolds to integrate complementary training signals and strengthen agentic coding capabilities. Furthermore, model merging combines different experts and checkpoints from different stages while mitigating conflicts among their parameter updates. The resulting IQuest-Q1 demonstrates capabilities in both coding and general tasks. We release IQuest-Q1 as an open-source model to support community research and development.