Awesome Ai Agents 2026
Section: Tracing and Monitoring · OSS LLM observability. One-line integration.
Entry
Appears in 9 awesome lists
Open-source LLM observability proxy (YC W23) with the largest open-source pricing database (300+ models). One-line proxy integration provides cost tracking, token monitoring, session tracing, and prompt versioning across providers. The AI Gateway component handles request routing and caching with…
Section: Tracing and Monitoring · OSS LLM observability. One-line integration.
Section: Links
Section: Prompts · 🧊 Open source LLM-Observability Platform for Developers. One-line integration for monitoring, metrics, evals, agent tracing, prompt management, playground, etc. Supports OpenAI SDK, Vercel AI SDK, Anthropic SDK, LiteLLM, LLamaIndex, LangChain, and more. 🍓 YC W23
Section: 5f · Observability + eval platforms (tracing · datasets · online/offline · CI) · https://github.com/helicone/helicone — OSS gateway + observability; "Scores" ingests external eval results.
Section: Open-source projects · GPT4 Monitoring and Observability.
Section: Observability & Tracing · Open-source LLM observability proxy (YC W23) with the largest open-source pricing database (300+ models). One-line proxy integration provides cost tracking, token monitoring, session tracing, and prompt versioning across providers. The AI Gateway component handles request routing and caching with…
Section: Observability · Open source LLM observability platform. One line of code to monitor, evaluate, and experiment with features like prompt management, agent tracing, and evaluations.
Section: 8. MLOps / LLMOps & Production · Open-source LLM observability platform with request logging, caching, prompt management, rate limiting, and cost analytics. Apache 2.0 licensed.
Section: Evaluation and Monitoring · Helicone is the all-in-one, open-source LLM developer platform.
🟢 — LLM eval + red-teaming/pentesting CLI with 50+ attack plugins (MIT). Note: OpenAI announced an acquisition agreement in March 2026; remains MIT-licensed — track governance. · updated 2026-10-13); Related: Promptfoo CI Action (checkout evaluation, not base/head diff; defaults to latest,…
Open-source AI observability & evaluation platform (Arize) — OpenTelemetry-native tracing for agents, LLM-as-judge evals, versioned datasets & experiments for prompt regression testing, prompt management with version control and replay, plus an MCP endpoint so Claude Code/Cursor can query traces…
The most widely adopted self-hostable LLM observability platform: traces every agent step, manages prompt versions, and runs evals in one tool. Preferred over cloud-only alternatives when data residency or cost control is a constraint.
🐙 Guides, papers, lecture, notebooks and resources for prompt engineering
Private AI platform for building intelligent agents and assistants with enterprise search. Features Agent Builder, deep research tools, multi-format document analysis, and multi-model support. MIT licensed.
DemoGPT enables you to create quick demos by just using prompt. It applies ToT approach on Langchain documentation tree.
🟢🟠 — Open AI gateway with provider routing, fallback and retry controls, guardrail integrations, observability, and MCP traffic support for model and agent applications. (Portkey) — note: the gateway is general infrastructure rather than a standalone security scanner; model-provider…