LiteLLM

Multi-provider LLM access

Give developers access to OpenAI, Azure, Gemini, Bedrock, and Anthropic models through a single gateway.

Cost tracking and chargebacks

Accurately charge teams for their LLM usage by attributing cost to keys, users, teams, or orgs.

Budget and rate limit management

Set budgets and rate limits (RPM/TPM) to control spending and prevent overuse.

LLM fallbacks

Automatically route requests to alternative models if the primary provider fails or is overloaded.

Observability and logging

Log spend to S3, GCS, or other storage, and integrate with observability tools like Langfuse, Arize Phoenix, Langsmith, and OpenTelemetry.

Prompt management

Manage and format prompts, including support for Hugging Face models.

Enterprise access control

Use JWT auth, SSO, and audit logs for secure, governed LLM access in large organizations.

Spend tracking

Attribute cost to key/user/team/org with automatic tracking across OpenAI, Azure, Bedrock, GCP, and other providers, plus tag-based spend tracking.

Budgets and rate limits

Set per-key or per-team budgets and enforce RPM/TPM limits to control usage.

OpenAI-compatible API

All requests use the OpenAI format, so developers don't need to transform inputs or outputs across providers.

LLM fallbacks

Configure automatic fallbacks to alternative models if the primary provider is unavailable.

Virtual keys and teams

Create virtual API keys, manage teams, and assign budgets at scale.

LLM guardrails

Apply guardrails to filter or modify LLM outputs for safety and compliance.

Batch API support

Process multiple requests in batch for efficiency.

Pass-through endpoints

Forward requests directly to underlying providers when needed.

Prompt management

Format prompts for different models, including Hugging Face models, without manual transformation.

S3 logging

Log all spend and usage data to S3, GCS, or other cloud storage for auditing.