When LLM Agents Meet Reinforcement Learning
Section: Environment · Tsinghua University
Entry
Appears in 8 awesome lists
Multi-environment agent benchmark (OS, DB, web, code) with a structured eval pipeline. Worth studying for its environment isolation design and task definition format when building custom eval environments for your harness.
Section: Environment · Tsinghua University
Section: Game (World Model & Agent) · A Comprehensive Benchmark to Evaluate LLMs as Agents.
Section: Benchmarks · 8-environment LLM agent benchmark.
Section: Evaluation & benchmarks · Comprehensive benchmark for LLM-as-Agent evaluation across 8 distinct environments. ICLR 2024.
Section: NLP · A Comprehensive Benchmark to Evaluate LLMs as Agents
Section: Verification & CI Integration · Multi-environment agent benchmark (OS, DB, web, code) with a structured eval pipeline. Worth studying for its environment isolation design and task definition format when building custom eval environments for your harness.
Section: Repositories · AgentBench v0.2 is a benchmark designed to evaluate Large Language Models as agents across a diverse set of environments, enhancing framework usability and extending model evaluations github
Section: Tools, APIs & MCP · A cross-environment benchmark spanning OS, databases, knowledge graphs, web browsing, and more, useful for seeing whether a harness generalizes beyond one narrow task loop.
Langchain integrates various providers like Anthropic, AWS, and OpenAI, and offers tools for components such as LLMs, chat models, and data analysis, supporting functionalities from Alpha Vantage to YouTube github | docs
(MIT) provides modules for structured outputs at different levels of abstraction, including output parsers for text completion endpoints, Pydantic programs for mapping prompts to structured outputs using function calling or output parsing, and pre-defined Pydantic programs for specific output types.
February 2026 release making human oversight a native workflow primitive: suspend execution at critical decision points, expose review-and-edit UI mid-flow, and route subsequent execution based on human action (approve/reject/escalate). Demonstrates how HITL transitions from bolt-on approval gates…
Microsoft's multi-agent conversation framework with a complete AgentChat layer covering agent loop, tool integration, termination conditions, and human-in-the-loop. The most comprehensive open-source reference for large-scale multi-agent harness design.
The open-source Python framework for real-time voice & multimodal conversational agents — every pipeline is an agent: multi-agent handoff, parallel fan-out, and sidecar workers over a shared bus, local or distributed; pluggable STT/LLM/TTS and transports (WebSocket/WebRTC), built-in Pipecat Flows…
The web-browsing agent module of the OpenAgents platform (HKU). Enables autonomous navigation of websites via natural language, as part of a larger multi-modal agent framework.
XAgent is an open-source, experimental Large Language Model-driven autonomous agent designed to autonomously solve a wide range of tasks with features like autonomy, safety, extensibility, a GUI for easy interaction, and the ability to cooperate with humans github | demo | docs | blog