Skip to content

Entry

LiteLLM

Appears in 16 awesome lists

Unified proxy and SDK that routes to 100+ LLM providers behind a single OpenAI-compatible interface, with a Router handling retry/fallback across deployments, per-project cost and rate-limit tracking, and OTEL callback integrations. The right infrastructure layer when your harness needs provider…

Open github.comberriai/litellm

Found in these lists

Awesome AI Coding Tools

Section: AI Frameworks and SDKs · Unified API proxy for 100+ LLM providers with load balancing, spend tracking, and rate limiting.

FreshScore 82

awesome-ChatGPT-repositories

Section: Langchain · lightweight package to simplify LLM API calls - Azure, OpenAI, Cohere, Anthropic, Replicate. Manages input/output translation

FreshScore 87

Awesome Cloud Native

Section: API Gateways & Proxies · Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq].

FreshScore 90

Awesome Data Analysis

Section: Tools · Unified interface to call all LLM APIs (OpenAI, Anthropic, Cohere, etc.) with consistent output formatting.

FreshScore 79

Awesome Generative AI Data Scientist

Section: Other · Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format.

StaleScore 54

Awesome Harness Engineering

Section: Task Runners & Orchestration · Unified proxy and SDK that routes to 100+ LLM providers behind a single OpenAI-compatible interface, with a Router handling retry/fallback across deployments, per-project cost and rate-limit tracking, and OTEL callback integrations. The right infrastructure layer when your harness needs provider…

FreshScore 93

Awesome Integration

Section: AI Gateway · An LLM gateway that exposes 100+ model providers through a unified OpenAI-compatible API, with spend tracking, fallbacks, and rate limiting.

FreshScore 84

Awesome LLM JSON List

Section: Python Libraries · (MIT) simplifies calling 100+ LLMs in the OpenAI format, supporting function calling, tool calling, and JSON mode.

StaleScore 54

Awesome LLM Resources

Section: 推理 Inference · Call all LLM APIs using the OpenAI format [Bedrock, Huggingface, VertexAI, TogetherAI, Azure, OpenAI, Groq etc.]

FreshScore 91

Awesome LLMOps

Section: LLMOps · A simple & light 100 line package to standardize LLM API calls across OpenAI, Azure, Cohere, Anthropic, Replicate API Endpoints

FreshScore 91

Awesome Open Source AI

Section: 8. MLOps / LLMOps & Production · AI Gateway to call 100+ LLM APIs in OpenAI format with unified cost tracking, guardrails, load balancing, and logging.

FreshScore 87

Awesome Production Machine Learning

Section: Deployment and Serving · LiteLLM is a Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq.

FreshScore 98

Awesome Prompts

Section: Tools & Libraries · Unified API for 100+ LLM providers

FreshScore 96

awesome-python

Section: LLM and Inference · The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

FreshScore 79

Awesome Jev

Section: Frameworks & integrations (89) · AI gateway with a TypeSafe guardrail and Jev classifier for policy judgments and complexity-based request routing.

FreshScore 85

Awesome AI Agents: Tools, Resources, and Projects

Section: Repositories · LiteLLM has added support for the OpenAI Assistants API, enabling seamless integration of stateful operations and automatic RAG pipelines into existing chatbots github

SlowScore 66

LangChain

Langchain integrates various providers like Anthropic, AWS, and OpenAI, and offers tools for components such as LLMs, chat models, and data analysis, supporting functionalities from Alpha Vantage to YouTube github | docs

In 22 listsDetails

LlamaIndex

(MIT) provides modules for structured outputs at different levels of abstraction, including output parsers for text completion endpoints, Pydantic programs for mapping prompts to structured outputs using function calling or output parsing, and pre-defined Pydantic programs for specific output types.

In 14 listsDetails

Ollama

Ollama is a tool for running large language models locally, offering easy setup for macOS, Windows, Linux, and Docker, along with a library of models and quickstart guides for customization and integration github | github profile

In 13 listsDetails

promptfoo

🟢 — LLM eval + red-teaming/pentesting CLI with 50+ attack plugins (MIT). Note: OpenAI announced an acquisition agreement in March 2026; remains MIT-licensed — track governance. · updated 2026-10-13); Related: Promptfoo CI Action (checkout evaluation, not base/head diff; defaults to latest,…

In 13 listsDetails

Open WebUI

Open WebUI is an extensible, feature-rich, and user-friendly self-hosted AI platform designed to operate entirely offline. It supports various LLM runners like Ollama and OpenAI-compatible APIs, with built-in inference engine for RAG, making it a powerful AI deployment solution.

In 12 listsDetails

Semantic Kernel

Semantic Kernel is an SDK that integrates Large Language Models (LLMs) like OpenAI, Azure OpenAI, and Hugging Face with conventional programming languages like C#, Python, and Java. Semantic Kernel achieves this by allowing you to define plugins that can be chained together in just a few lines of…

In 11 listsDetails

Flowise

Flowise simplifies the creation of applications leveraging large language models (LLMs) by providing a drag-and-drop interface for customizing AI workflows, offering easy installation, Docker support, development tools, and documentation for integrating various functionalities such as…

In 12 listsDetails

MarkItDown

Python tool for converting files and office documents to Markdown. Supports PDF, PowerPoint, Word, Excel, images, audio, HTML, and more with OCR and transcription capabilities. MIT licensed.

In 9 listsDetails