daVinci-Prover uses agent-native mid-training for formal theorem proving
Original titledaVinci-Prover: Agent-Native Mid-Training for Formal Theorem Proving
- Outlet
- Takara TLDR
- Published
- Importance
- 3/5
Summary
The authors present daVinci-Prover, which applies agent-native mid-training to improve formal theorem-proving models. They synthesize a 33.7B-token corpus covering decomposition, search, self-verification, and revision, achieving 13.7-point average improvement over a Qwen3-8B baseline on MiniF2F, FATE-M, and ProofNet benchmarks.
Original excerpt
Advances in large language models (LLMs) are rapidly transforming formal theorem proving. Yet further progress remains constrained by the lack of a transparent and effective data recipe for systematically improving theorem-proving models. In this work, we argue that effective agentic provers require strong mathematical skill priors to ground theorem proving and agentic trajectories to enable long-horizon, interactive proof construction. To this end, we synthesize an agent-native 33.7B-token mid-training corpus covering decomposition, search, self-verification, and revision, together with verifier-grounded agentic trajectories. We further introduce the daVinci-Prover harness, which enables models to construct proofs and interact with a Lean verifier in an isolated workspace, and use it to collect approximately 36K high-quality agent trajectories for supervised fine-tuning. Our mid-training recipe yields an average improvement of 13.7 points over the matched Qwen3-8B SFT-only baseline across MiniF2F, FATE-M, and ProofNet. When applied to the Qwen3.5 series, our approach also yields strong performance across theorem-proving benchmarks and achieves near-saturating MiniF2F performance with memory compaction. The results highlight agent-native mid-training as an effective foundation for developing stronger interactive theorem-proving agents.