Skip to content

Entry

OpenRLHF

Appears in 6 awesome lists

The Ray+vLLM pioneer of open RLHF/RL post-training — the first framework built around a fully decoupled Ray-based distributed architecture (vLLM generation + DeepSpeed ZeRO/FSDP training, placement-colocated hybrid engine with sleep-mode weight sharing between actor/critic/reward/reference); PPO /…

Open github.comopenrlhf/openrlhf

Found in these lists

When LLM Agents Meet Reinforcement Learning

Section: Base Framework · OpenRLHF

FreshScore 88

Awesome Agent Evals

Section: 5e · RL-environment / verifiable-reward toolkits (eval ⇄ training) · https://github.com/OpenRLHF/OpenRLHF · SkyRL — https://github.com/NovaSky-AI/SkyRL · AReaL — https://github.com/areal-project/AReaL · ROLL — https://github.com/alibaba/ROLL · rLLM — https://github.com/agentica-project/rllm · TRL — https://github.com/huggingface/trl — the RL-training stack agents…

FreshScore 87

Awesome local LLM

Section: Training and Fine-tuning · an easy-to-use, high-performance open-source RLHF framework built on Ray, vLLM, ZeRO-3 and HuggingFace Transformers, designed to make RLHF training simple and accessible

FreshScore 90

Awesome Open Source AI

Section: 7. Training & Fine-tuning Ecosystem · Easy-to-use, scalable RLHF framework based on Ray. Supports PPO, GRPO, REINFORCE++, DAPO with vLLM integration and async training. Apache 2.0 licensed.

FreshScore 87

Awesome Production Machine Learning

Section: Industry Strength Reinforcement Learning · OpenRLHF is an open-source framework for reinforcement learning from human feedback (RLHF).

FreshScore 98

Awesome Prompts

Section: Agent Training (RL Post-Training) · The Ray+vLLM pioneer of open RLHF/RL post-training — the first framework built around a fully decoupled Ray-based distributed architecture (vLLM generation + DeepSpeed ZeRO/FSDP training, placement-colocated hybrid engine with sleep-mode weight sharing between actor/critic/reward/reference); PPO /…

FreshScore 96

TRL

Hugging Face's post-training library — the reference implementation of GRPO (the algorithm DeepSeek popularized) and the most-used single-GPU-to-cluster starting point for RL post-training; 75+ trainers across SFT / DPO / KTO / PPO / ORPO / CPO and more, with vLLM generation, PEFT/LoRA, and AMD…

In 10 listsDetails

slime

Zhipu AI / Tsinghua's (THUDM) RL scaling framework — the production trainer behind the GLM series from GLM-4.5 to GLM-5.3-Flash; connects Megatron training with SGLang rollout through a unified train–rollout–buffer architecture (parameters sync, router-based generation, prompt/verifier management)…

In 6 listsDetails

Kiln

Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.

In 5 listsDetails

NeMo-RL

NVIDIA's scalable RL post-training library (successor to NeMo Aligner, re-architected with the NeMo Framework) — the megatron-core-backed option in the open stack: GRPO / GSPO / DAPO / PPO / DPO / SFT(+LoRA) / on-policy distillation / GDPO with training via PyTorch DTensor (FSDP2) or Megatron Core…

In 5 listsDetails

prime-rl

Prime Intellect's fully asynchronous RL framework — "agentic RL training at scale" (the INTELLECT-3 report) — built to train 1T+ parameter MoE models on 1,000+ GPUs through three abstractions: a lightweight CPU Orchestrator (dataflow/scheduling), an FSDP2 Trainer, and vLLM inference as…

In 5 listsDetails

verl

The most widely adopted open-source RL post-training framework for LLMs (ByteDance Seed; HybridFlow architecture, EuroSys 2025) — a flexible single-controller/dataflow abstraction that hybrid-orchestrates rollout generation (vLLM/SGLang) and training (FSDP/Megatron) in one program; PPO / GRPO /…

In 5 listsDetails

sentence-transformers

a Python library for using and training embedding and reranker models for applications like retrieval augmented generation, semantic search, and more

In 4 listsDetails

agent-lightning

Microsoft Research's harnessed agentic RL trainer — v1.0 (~3,500 LOC, arXiv 2608.17528) trains agents inside their real production harnesses instead of reimplementing them within the trainer: an OpenAI-compatible API Gateway proxy records prompts/responses/logprobs of any agent (LangGraph, OpenAI…

In 4 listsDetails