Skip to content

Entry

Reef

Appears in 4 awesome lists

Reef serves agent traffic behind OpenAI- and Anthropic-compatible endpoints and returns a receipt for each interaction, so a later report can attach a score or feedback to it; eligible records are batched into a candidate update to model weights or to an agent harness of prompts, rules and skills,…

Open github.comhuman-agent-society/reef

Found in these lists

Awesome Production Machine Learning

Section: Deployment and Serving · Reef serves agent traffic behind OpenAI- and Anthropic-compatible endpoints and returns a receipt for each interaction, so a later report can attach a score or feedback to it; eligible records are batched into a candidate update to model weights or to an agent harness of prompts, rules and skills,…

FreshScore 98

Awesome Prompts

Section: Automatic Prompt Optimization · Open-source infrastructure for continually self-improving agents — connects agent inference, feedback, learning, and versioned delivery; either train model weights (Slime + SGLang) or improve the agent harness itself (prompts, rules, skills) from interaction feedback, no local GPUs required;…

FreshScore 96

Awesome RSI (Recursive Self-Improvement)

Section: Harness / Memory / Skill Evolution · Continual-learning serving infrastructure that records agent interactions, turns matched feedback into model-weight or harness updates, and publishes accepted updates as versioned artifacts.

FreshScore 85

Best Of Agent Harnesses

Section: On the radar · Infrastructure for continually self-improving agents: records live traffic and feedback, then ships a scored update to model weights or the harness.

FreshScore 80

LiteLLM

Unified proxy and SDK that routes to 100+ LLM providers behind a single OpenAI-compatible interface, with a Router handling retry/fallback across deployments, per-project cost and rate-limit tracking, and OTEL callback integrations. The right infrastructure layer when your harness needs provider…

In 16 listsDetails

LocalAI

robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video,…

In 14 listsDetails

Open WebUI

Open WebUI is an extensible, feature-rich, and user-friendly self-hosted AI platform designed to operate entirely offline. It supports various LLM runners like Ollama and OpenAI-compatible APIs, with built-in inference engine for RAG, making it a powerful AI deployment solution.

In 12 listsDetails

llama.cpp

Pure C/C++ inference engine with GGUF format support. The gold standard for CPU/GPU/Apple Silicon on-device running. Includes llama-server for OpenAI-compatible API. Now at 100K+ stars.

In 12 listsDetails

vLLM

State-of-the-art serving engine with PagedAttention and continuous batching. Currently the fastest production-grade LLM server.

In 11 listsDetails

Bifrost

🟢🟠 — Apache-licensed AI and MCP gateway with multi-provider routing, virtual-key access controls, budgets, rate limits, MCP aggregation, OAuth, automatic fallbacks, and load balancing. (Maxim) — note: model-provider credentials and proxied request/response data are sensitive; restrict and…;…

In 11 listsDetails

OpenLLM

Production-grade platform for running any open-source LLMs as OpenAI-compatible API endpoints. Supports 50+ models with built-in streaming, batching, and auto-acceleration. Apache 2.0 licensed.

In 10 listsDetails

SGLang

(MPL-2.0) allows specifying JSON schemas using regular expressions or Pydantic models for constrained decoding. Its high-performance runtime accelerates JSON decoding.

In 9 listsDetails