Awesome Agent Evals
Section: 5f · Observability + eval platforms (tracing · datasets · online/offline · CI) · https://www.fiddler.ai/ — 🆕 Trust Models (Safety/PII/Faithfulness) scoring in <100ms; Guardrails + agentic observability.
FreshScore 88
Entry
Fiddler AI
Appears in 4 awesome lists
The all-in-one AI Observability and Security platform for responsible AI. It provides monitoring, analytics, and centralized controls to operationalize ML, GenAI, and LLM applications with trust. Fiddler helps enterprises scale LLM and ML deployments to deliver high performance AI, reduce costs,…
Found in these lists
Awesome Machine Learning
Section: Tools · The all-in-one AI Observability and Security platform for responsible AI. It provides monitoring, analytics, and centralized controls to operationalize ML, GenAI, and LLM applications with trust. Fiddler helps enterprises scale LLM and ML deployments to deliver high performance AI, reduce costs,…
FreshScore 94
Awesome MLOps
Section: Visual Analysis and Debugging · Monitor, explain, and analyze your AI in production.
FreshScore 88
Awesome OSINT List
Section: LLM Security & AI Security · ML model performance monitoring and explainability platform.
FreshScore 88
Often listed next to it
Opik
Comet's open-source AI observability and evaluation platform: deep tracing of LLM calls, conversation logging, and agent activity, plus built-in eval metrics, prompt versioning, guardrails, and the Opik Agent Optimizer. Worth including because it unifies observability, verification, and…
In 18 listsDetails
Netron
Visualizer for deep learning and machine learning models (no Python code, but visualizes models from most Python Deep Learning frameworks).
In 16 listsDetails
m2cgen
Transpile trained ML models into other languages. sklearn-porter - Transpile trained scikit-learn estimators to C, Java, JavaScript and others. mlflow - Manage the machine learning lifecycle, including experimentation, reproducibility and deployment. skll - Command-line utilities to make it easier…
In 16 listsDetails
promptfoo
🟢 — LLM eval + red-teaming/pentesting CLI with 50+ attack plugins (MIT). Note: OpenAI announced an acquisition agreement in March 2026; remains MIT-licensed — track governance. · updated 2026-10-13); Related: Promptfoo CI Action (checkout evaluation, not base/head diff; defaults to latest,…
In 13 listsDetails
Phoenix
Open-source AI observability & evaluation platform (Arize) — OpenTelemetry-native tracing for agents, LLM-as-judge evals, versioned datasets & experiments for prompt regression testing, prompt management with version control and replay, plus an MCP endpoint so Claude Code/Cursor can query traces…
In 12 listsDetails
Langfuse
The most widely adopted self-hostable LLM observability platform: traces every agent step, manages prompt versions, and runs evals in one tool. Preferred over cloud-only alternatives when data residency or cost control is a constraint.
In 11 listsDetails
Helicone
Open-source LLM observability proxy (YC W23) with the largest open-source pricing database (300+ models). One-line proxy integration provides cost tracking, token monitoring, session tracing, and prompt versioning across providers. The AI Gateway component handles request routing and caching with…
In 9 listsDetails
Future AGI
🟢🟠 — Self-hostable evaluation and tracing platform with an OpenAI-compatible Go gateway for inline model-traffic checks, MCP/tool permissions and optional external guardrail services. (Future AGI) Caveat: the core is Apache-2.0; enterprise components have a separate commercial license. Gateway…;…
In 9 listsDetails