LiteLLM, an open-source AI gateway for 100+ LLMs
Original titleBerriAI/litellm
- Outlet
- GitHub Trending
- Published
- Importance
- 2/5
Summary
LiteLLM is an open-source AI gateway that provides a unified interface to call over 100 LLM providers including OpenAI, Anthropic, and Bedrock in OpenAI format. It offers cost tracking, guardrails, load balancing, and logging for production deployments.
Also reported by 1 outlet
Original excerpt
The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM] https://docs.litellm.ai/docs/ 🚅 LiteLLM LiteLLM AI Gateway Open Source AI Gateway for 100+ LLMs. Self-hosted. Enterprise-ready. Call any LLM in OpenAI format. LiteLLM Proxy Server (AI Gateway) | Hosted Proxy | Enterprise Tier | Website What is LiteLLM LiteLLM is an open source AI Gateway that gives you a single, unified interface to call 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure, and more — using the OpenAI format. Use it as a Python SDK for direct library integration, or deploy the AI Gateway (Proxy Server) as a centralized service for your team or organization. Jump to LiteLLM Proxy (LLM Gateway) Docs Jump to Supported LLM Providers Why LiteLLM Managing LLM calls across providers gets complicated fast — different SDKs, auth patterns, request formats, and error types for every model. LiteLLM removes that friction: Unified API — one interface for 100+ LLMs, no provider-specific SDK juggling Drop-in OpenAI compatibility — swap providers without rewriting your code Production-ready gateway — virtual keys, spend tracking, guardrails, load balancing, and an admin dashboard out of the box 8ms P95 latency at 1k RPS (benchmarks) OSS Adopters Netflix Features LLMs - Call 100+ LLMs (Python SDK + AI Gateway) All Supported Endpoints - /chat/completions, /responses, /embeddings, /images, /audio, /batches, /rerank, /a2a, /messages and more. Python SDK uv add litellm An independent litellm-core distribution provides the Python SDK with the same import litellm API and runtime dependencies. It has no optional extras, CLI entry points, or bundled dashboard. Install one SDK distribution per environment because litellm and litellm-core own overlapping Python files. Use litellm for the proxy, CLI, and optional extras Build and in