Entry
Appears in 4 awesome lists
🟢⚠️ — Safety tooling including Llama Guard classifiers, CodeShield, CyberSecEval, and LlamaFirewall's configurable prompt, code, and alignment scanner pipelines. (Meta) — note: LlamaFirewall's MIT code license does not cover all bundled projects or model weights; model-backed checks require…;…
Section: Scanners, Evals & Guardrails · 🟢⚠️ — Safety tooling including Llama Guard classifiers, CodeShield, CyberSecEval, and LlamaFirewall's configurable prompt, code, and alignment scanner pipelines. (Meta) — note: LlamaFirewall's MIT code license does not cover all bundled projects or model weights; model-backed checks require…;…
Section: 8. MLOps / LLMOps & Production · Comprehensive set of tools to assess and improve LLM security. Includes Llama Guard safety classifiers, CyberSec Eval benchmarks, and Prompt Guard for prompt injection detection. BSD-3-Clause licensed.
Section: AI & Machine Learning Security · Meta's open-source LLM security toolkit including Llama Guard classifiers, CodeShield for insecure code detection, and CyberSecEval for benchmarking AI safety and security.
Section: Tools · Set of tools to assess and improve LLM security.
Comet's open-source AI observability and evaluation platform: deep tracing of LLM calls, conversation logging, and agent activity, plus built-in eval metrics, prompt versioning, guardrails, and the Opik Agent Optimizer. Worth including because it unifies observability, verification, and…
Unified proxy and SDK that routes to 100+ LLM providers behind a single OpenAI-compatible interface, with a Router handling retry/fallback across deployments, per-project cost and rate-limit tracking, and OTEL callback integrations. The right infrastructure layer when your harness needs provider…
by Continuum AI - Records a Claude Code run through a local proxy and replays it offline, reproducing it byte-for-byte with the network off, or forking it from any checkpoint onto a different model. Capture happens below the harness, so shell exit codes, per-turn file changes and MCP calls land on…
🟢 — LLM eval + red-teaming/pentesting CLI with 50+ attack plugins (MIT). Note: OpenAI announced an acquisition agreement in March 2026; remains MIT-licensed — track governance. · updated 2026-10-13); Related: Promptfoo CI Action (checkout evaluation, not base/head diff; defaults to latest,…
Cloud-native orchestration platform for developing and maintaining data assets including ML models. Declarative programming model with integrated lineage and observability. Apache 2.0 licensed.
Open-source AI observability & evaluation platform (Arize) — OpenTelemetry-native tracing for agents, LLM-as-judge evals, versioned datasets & experiments for prompt regression testing, prompt management with version control and replay, plus an MCP endpoint so Claude Code/Cursor can query traces…
Workflow management system that makes it easy to take your data pipelines and add semantics like retries, logging, dynamic mapping, caching, failure notifications, and more.
Multi-cluster Kubernetes MCP server bridging Gemini CLI to kubeconfig and Kubernetes APIs. Manage clusters, policies, and 20+ CNCF project integrations across edge and cloud. Install via brew tap kubestellar/tap && brew install kc-agent.