Awesome LLMOps
Section: LLMOps · Cost-effective LLM development in any cloud (AWS, GCP, Azure, Lambda, etc).
ActiveScore 75
Entry
dstack
Appears in 4 awesome lists
Vendor-agnostic orchestration for training, inference and agentic workloads across NVIDIA, AMD, TPU, and Tenstorrent on clouds, Kubernetes, and bare metal. MPL-2.0 licensed.
Open github.comdstackai/dstack
Found in these lists
Awesome MLOps
Section: Workflow Tools · An open-core tool to automate data and training workflows.
FreshScore 80
Awesome Open Source AI
Section: 7. Training & Fine-tuning Ecosystem · Vendor-agnostic orchestration for training, inference and agentic workloads across NVIDIA, AMD, TPU, and Tenstorrent on clouds, Kubernetes, and bare metal. MPL-2.0 licensed.
FreshScore 89
Awesome Production Machine Learning
Section: Model Training and Orchestration · dstack is an open-source container orchestrator that simplifies workload orchestration and drives GPU utilization for ML teams.
FreshScore 92
Often listed next to it
LiteLLM
Unified proxy and SDK that routes to 100+ LLM providers behind a single OpenAI-compatible interface, with a Router handling retry/fallback across deployments, per-project cost and rate-limit tracking, and OTEL callback integrations. The right infrastructure layer when your harness needs provider…
In 16 listsDetails
Opik
Comet's open-source AI observability and evaluation platform: deep tracing of LLM calls, conversation logging, and agent activity, plus built-in eval metrics, prompt versioning, guardrails, and the Opik Agent Optimizer. Worth including because it unifies observability, verification, and…
In 16 listsDetails
Luigi
Python module for building complex pipelines of batch jobs. Handles dependency resolution, workflow management, visualization, and Hadoop integration. Built at Spotify and battle-tested in production. Apache 2.0 licensed.
In 15 listsDetails
Colossal-AI
(from Hpcaitech) - A Unified Deep Learning System for Large-Scale Parallel Training (1D, 2D, 2.5D, 3D and sequence parallelism, and ZeRO protocol).
In 14 listsDetails
Dify
February 2026 release making human oversight a native workflow primitive: suspend execution at critical decision points, expose review-and-edit UI mid-flow, and route subsequent execution based on human action (approve/reject/escalate). Demonstrates how HITL transitions from bolt-on approval gates…
In 14 listsDetails
Ray
A fast and simple framework for building and running distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning library. ray.io
In 13 listsDetails
Haystack
Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search,…
In 13 listsDetails
FastChat
Open platform for training, serving, and evaluating large language model chatbots. Powers Chatbot Arena (lmarena.ai) serving 10M+ requests for 70+ LLMs. Includes training code for Vicuna, MT-Bench evaluation, and distributed multi-model serving with OpenAI-compatible APIs. Apache 2.0 licensed.
In 12 listsDetails