Skip to content

Entry

Helicone

Appears in 9 awesome lists

Open-source LLM observability proxy (YC W23) with the largest open-source pricing database (300+ models). One-line proxy integration provides cost tracking, token monitoring, session tracing, and prompt versioning across providers. The AI Gateway component handles request routing and caching with…

Open github.comhelicone/helicone

Found in these lists

Awesome Ai Agents 2026

Section: Tracing and Monitoring · OSS LLM observability. One-line integration.

ActiveScore 70

Awesome Ai Sdks

Section: Links

ActiveScore 69

awesome-ChatGPT-repositories

Section: Prompts · 🧊 Open source LLM-Observability Platform for Developers. One-line integration for monitoring, metrics, evals, agent tracing, prompt management, playground, etc. Supports OpenAI SDK, Vercel AI SDK, Anthropic SDK, LiteLLM, LLamaIndex, LangChain, and more. 🍓 YC W23

FreshScore 87

Awesome Agent Evals

Section: 5f · Observability + eval platforms (tracing · datasets · online/offline · CI) · https://github.com/helicone/helicone — OSS gateway + observability; "Scores" ingests external eval results.

FreshScore 87

Awesome GPT-4

Section: Open-source projects · GPT4 Monitoring and Observability.

StaleScore 48

Awesome Harness Engineering

Section: Observability & Tracing · Open-source LLM observability proxy (YC W23) with the largest open-source pricing database (300+ models). One-line proxy integration provides cost tracking, token monitoring, session tracing, and prompt versioning across providers. The AI Gateway component handles request routing and caching with…

FreshScore 93

Awesome LLMOps

Section: Observability · Open source LLM observability platform. One line of code to monitor, evaluate, and experiment with features like prompt management, agent tracing, and evaluations.

FreshScore 91

Awesome Open Source AI

Section: 8. MLOps / LLMOps & Production · Open-source LLM observability platform with request logging, caching, prompt management, rate limiting, and cost analytics. Apache 2.0 licensed.

FreshScore 87

Awesome Production Machine Learning

Section: Evaluation and Monitoring · Helicone is the all-in-one, open-source LLM developer platform.

FreshScore 98

promptfoo

🟢 — LLM eval + red-teaming/pentesting CLI with 50+ attack plugins (MIT). Note: OpenAI announced an acquisition agreement in March 2026; remains MIT-licensed — track governance. · updated 2026-10-13); Related: Promptfoo CI Action (checkout evaluation, not base/head diff; defaults to latest,…

In 13 listsDetails

Phoenix

Open-source AI observability & evaluation platform (Arize) — OpenTelemetry-native tracing for agents, LLM-as-judge evals, versioned datasets & experiments for prompt regression testing, prompt management with version control and replay, plus an MCP endpoint so Claude Code/Cursor can query traces…

In 12 listsDetails

Langfuse

The most widely adopted self-hostable LLM observability platform: traces every agent step, manages prompt versions, and runs evals in one tool. Preferred over cloud-only alternatives when data residency or cost control is a constraint.

In 11 listsDetails

Prompt Engineering Guide

🐙 Guides, papers, lecture, notebooks and resources for prompt engineering

In 13 listsDetails

DocsGPT

Private AI platform for building intelligent agents and assistants with enterprise search. Features Agent Builder, deep research tools, multi-format document analysis, and multi-model support. MIT licensed.

In 10 listsDetails

DemoGPT

DemoGPT enables you to create quick demos by just using prompt. It applies ToT approach on Langchain documentation tree.

In 9 listsDetails

Portkey AI Gateway

🟢🟠 — Open AI gateway with provider routing, fallback and retry controls, guardrail integrations, observability, and MCP traffic support for model and agent applications. (Portkey) — note: the gateway is general infrastructure rather than a standalone security scanner; model-provider…

In 7 listsDetails

Rapid-MLX

The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Drop-in OpenAI replacement. Works with Claude Code, Cursor, Aider.

In 6 listsDetails