Skip to content

Entry

traceAI

Appears in 4 awesome lists

Apache-2.0 OpenTelemetry instrumentation for 50+ AI frameworks across Python, TypeScript, Java and C# that records LLM calls, tool calls, retrieval and agent steps as standard spans, so harness traces stay portable to any OTel backend.

Open github.comfuture-agi/traceai

Found in these lists

Awesome LangChain

Section: Platforms · Open-source OpenTelemetry-native tracing framework for LLM applications, with native LangChain instrumentation and support for 20+ frameworks.

FreshScore 94

Awesome LLMOps

Section: Observability · Open-source AI tracing framework built on OpenTelemetry for deep observability across agentic and LLM workflows.

FreshScore 91

Awesome OpenTelemetry

Section: Open Source · Open-source OpenTelemetry-native instrumentation framework for AI applications that auto-instruments 20+ LLM providers and agent frameworks, capturing prompts, tokens, latency, and errors with zero code changes.

FreshScore 79

Awesome Harness Engineering

Section: Telemetry, Tracing & Performance · Apache-2.0 OpenTelemetry instrumentation for 50+ AI frameworks across Python, TypeScript, Java and C# that records LLM calls, tool calls, retrieval and agent steps as standard spans, so harness traces stay portable to any OTel backend.

FreshScore 93

Langfuse

LLM engineering platform for model tracing, prompt management, and application evaluation. Langfuse helps teams collaboratively debug, analyze, and iterate on their LLM applications such as chatbots or AI agents. (Demo, Source Code, Clients) MIT Docker

In 10 listsDetails

Deepchecks

Validation & testing of machine learning models and data during model development, deployment, and production. This includes checks and suites related to various types of issues, such as model performance, data integrity, distribution mismatches, and more.

In 9 listsDetails

Helicone

Open-source LLM observability proxy (YC W23) with the largest open-source pricing database (300+ models). One-line proxy integration provides cost tracking, token monitoring, session tracing, and prompt versioning across providers. The AI Gateway component handles request routing and caching with…

In 9 listsDetails

Evidently

Interactive reports to analyze machine learning models during validation or production monitoring.

In 8 listsDetails

OpenLLMetry

OpenTelemetry-based instrumentation for LLM calls and agent steps: adds trace spans to every inference and tool call without modifying business logic. The cleanest way to bring the existing OTEL ecosystem (Grafana, Datadog, Jaeger) to a harness.

In 8 listsDetails

Maxim AI

The agent simulation, evaluation, and observability platform helping product teams ship their AI applications with the quality and speed needed for real-world use.

In 9 listsDetails

CodeBurn

Free, local tool that tracks AI coding token usage and cost across 31 tools and agents by model, project, and task. Worth including because cross-tool cost visibility is the missing prerequisite for agent FinOps in multi-tool teams — most observability tools either require cloud upload or only…

In 6 listsDetails

onWatch

onWatch is a lightweight Go CLI that tracks AI API quota usage across multiple providers (Anthropic Pro/Max Plans, Codex, Gemini CLI, Synthetic, Z.ai, GitHub Copilot, MiniMax Coding/Token Plan, Antigravity, OpenRouter) in real time, with consumption rate projections, historical usage graphs, and…

In 6 listsDetails