Microsoft, 1비트 LLM 추론 프레임워크 BitNet 공개
microsoft/BitNet
- 매체
- GitHub Trending
- 게시
- 중요도
- 2/5
- 화제 지수
- —
한국어 요약
Microsoft가 1비트 LLM 추론 프레임워크 bitnet.cpp를 공개했다. VibeASR.cpp 실시간 다국어 음성 인식 엔진을 BitNet I2_S 양자화로 CPU에서 구현한다.
원문 발췌 · English
Official inference framework for 1-bit LLMs bitnet.cpp 📰 News 07/23/2026: 📣 We released VibeASR.cpp — a real-time multilingual ASR inference engine on CPU using BitNet I2_S quantization, achieving RTF =3.10 cmake>=3.22 clang>=18 For Windows users, install Visual Studio 2022. In the installer, toggle on at least the following options(this also automatically installs the required additional tools like CMake): Desktop-development with C++ C++-CMake Tools for Windows Git for Windows C++-Clang Compiler for Windows MS-Build Support for LLVM-Toolset (clang) For Debian/Ubuntu users, you can download with Automatic installation script bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" conda (highly recommend) Build from source Important If you are using Windows, please remember to always use a Developer Command Prompt / PowerShell for VS2022 for the following commands. Please refer to the FAQs below if you see any issues. Clone the repo git clone --recursive https://github.com/microsoft/BitNet.git cd BitNet Install the dependencies # (Recommended) Create a new conda environment conda create -n bitnet-cpp python=3.10 conda activate bitnet-cpp pip install -r requirements.txt Build the project # Manually download the model and run with local path huggingface-cli download microsoft/BitNet-b1.58-2B-4T-gguf --local-dir models/BitNet-b1.58-2B-4T python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s usage: setup_env.py [-h] [--hf-repo {1bitLLM/bitnet_b1_58-large,1bitLLM/bitnet_b1_58-3B,HF1BitLLM/Llama3-8B-1.58-100B-tokens,tiiuae/Falcon3-1B-Instruct-1.58bit,tiiuae/Falcon3-3B-Instruct-1.58bit,tiiuae/Falcon3-7B-Instruct-1.58bit,tiiuae/Falcon3-10B-Instruct-1.58bit}] [--model-dir MODEL_DIR] [--log-dir LOG_DIR] [--quant-type {i2_s,tl1}] [--quant-embd] [--use-pretuned] Setup the environment for running inference optional arguments: -h, --help show this help message and exit --hf-repo {1bitLLM/bitnet_b1_58-large,1bitLLM/bitnet_b1_58-3B,HF1BitLLM/Llama3-8B-1.58-100B-tokens,tiiuae/Falc