awesome-ChatGPT-repositories
Section: Openai · Uncomplicated Observability for Python and beyond! 🪵🔥
Entry
Appears in 5 awesome lists
AI observability platform from the Pydantic team with a unique angle: all trace data is SQL-queryable (PostgreSQL-compatible), so coding agents can query production observability data directly via the Logfire MCP server. Full-stack OTEL tracing covers both the AI layer and backend — letting you…
Section: Openai · Uncomplicated Observability for Python and beyond! 🪵🔥
Section: Observability & Tracing · AI observability platform from the Pydantic team with a unique angle: all trace data is SQL-queryable (PostgreSQL-compatible), so coding agents can query production observability data directly via the Logfire MCP server. Full-stack OTEL tracing covers both the AI layer and backend — letting you…
Section: GenAI / LLM Instrumentation · OpenTelemetry-native observability platform and SDK from the Pydantic team. Auto-instruments Pydantic AI, OpenAI, Anthropic, LangChain and other AI frameworks, emitting standard OTel spans consumable by any OTLP-compatible backend.
Section: Monitoring and Observability · AI observability platform for production LLM and agent systems.
Section: Logging · The observability platform for Python, from the makers of Pydantic.
Comet's open-source AI observability and evaluation platform: deep tracing of LLM calls, conversation logging, and agent activity, plus built-in eval metrics, prompt versioning, guardrails, and the Opik Agent Optimizer. Worth including because it unifies observability, verification, and…
industrial-grade ASR toolkit; 170× realtime on GPU, 50+ languages, built-in VAD, punctuation, speaker diarization, and emotion detection. Includes non-autoregressive SenseVoice and LLM-based Fun-ASR-Nano models.
Open-source AI observability & evaluation platform (Arize) — OpenTelemetry-native tracing for agents, LLM-as-judge evals, versioned datasets & experiments for prompt regression testing, prompt management with version control and replay, plus an MCP endpoint so Claude Code/Cursor can query traces…
The most widely adopted self-hostable LLM observability platform: traces every agent step, manages prompt versions, and runs evals in one tool. Preferred over cloud-only alternatives when data residency or cost control is a constraint.
Open-source LLM observability proxy (YC W23) with the largest open-source pricing database (300+ models). One-line proxy integration provides cost tracking, token monitoring, session tracing, and prompt versioning across providers. The AI Gateway component handles request routing and caching with…
🟢🟠 — Self-hostable evaluation and tracing platform with an OpenAI-compatible Go gateway for inline model-traffic checks, MCP/tool permissions and optional external guardrail services. (Future AGI) Caveat: the core is Apache-2.0; enterprise components have a separate commercial license. Gateway…;…
AutoGPT provides accessible AI tools for building and using AI agents, offering a comprehensive framework including Forge for agent creation, agbenchmark for performance evaluation, a leaderboard for competition, a user-friendly UI, and CLI for seamless integration and management github | github…
https://github.com/openai/evals — the completion_fn abstraction = swap the system-under-test. (Best-practices: https://developers.openai.com/api/docs/guides/evaluation-best-practices)