Skip to content

Entry

llama.cpp

Appears in 12 awesome lists

Pure C/C++ inference engine with GGUF format support. The gold standard for CPU/GPU/Apple Silicon on-device running. Includes llama-server for OpenAI-compatible API. Now at 100K+ stars.

Open github.comggml-org/llama.cpp

Found in these lists

Awesome Ai Agents 2026

Section: Local LLM Runners · C/C++ inference. CPU, GPU, Apple Silicon. Foundation of local AI.

ActiveScore 70

Awesome Stars

Section: C++ · LLM inference in C/C++

SlowScore 56

awesome-jev

Section: Infra / SDKs / Integrations · Runtime support: merged support for d3 decision models in llama.cpp, which puts the model class beside ordinary inference in one of the most widely deployed runtimes.

FreshScore 85

Awesome local LLM

Section: Inference engines · LLM inference in C/C++

FreshScore 90

Awesome Model Quantization

Section: Inference and hardware · Local LLM inference with GGUF models and multiple quantization formats.

FreshScore 85

Awesome Open Source AI

Section: 3. Inference Engines & Serving · Pure C/C++ inference engine with GGUF format support. The gold standard for CPU/GPU/Apple Silicon on-device running. Includes llama-server for OpenAI-compatible API. Now at 100K+ stars.

FreshScore 87

Awesome Production Machine Learning

Section: Deployment and Serving · llama.cpp is an open source software library that performs inference on various large language models such as Llama.

FreshScore 98

CPU Performance Engineering

Section: Runtimes · Where new quantization types, kernels and thread pools land first, each with the perplexity and speed table behind it.

FreshScore 82

Awesome Privacy

Section: Artificial Intelligence · C/C++ engine that runs open language models locally on ordinary CPUs and GPUs. MIT

FreshScore 98

Awesome Generative AI

Section: Developer tools · Inference of Meta's LLaMA model (and others) in pure C/C++. #opensource

FreshScore 90

awesome-c

Section: LLM and Inference · LLM inference in C/C++

FreshScore 83

awesome-cpp

Section: LLM and Inference · LLM inference in C/C++

FreshScore 77

LocalAI

robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video,…

In 14 listsDetails

PydanticAI

June 2026 harness-first redesign built around the Capability primitive: a single composable unit bundling instructions, tools, lifecycle hooks, and model settings. The split between a small stable core and a fast-moving pydantic-ai-harness lets capabilities graduate as they prove essential, while…

In 13 listsDetails

Ollama

Ollama is a tool for running large language models locally, offering easy setup for macOS, Windows, Linux, and Docker, along with a library of models and quickstart guides for customization and integration github | github profile

In 13 listsDetails

vLLM

State-of-the-art serving engine with PagedAttention and continuous batching. Currently the fastest production-grade LLM server.

In 11 listsDetails

GPTCache

@ferret_db @ossinsight @milvusio we'd love to get a ferretdb backend for gptcache. we'll add it to the todo list.and on behalf of milvus developers and the broader community, thanks for the shout-out!

In 9 listsDetails

LM Studio

LM Studio offers a platform for running various local LLMs like LLaMa, Falcon, MPT, and others offline, featuring a Chat UI, OpenAI-compatible server, and model downloads from Hugging Face, with support for Mac, Windows, and Linux, emphasizing privacy and no data collection, free for personal use…

In 11 listsDetails

gpt4all

is an ecosystem of open-source chatbots trained on a massive collections of clean assistant data including code, stories and dialogue based on LLaMa.

In 11 listsDetails

Jan

Jan is an open-source, development-stage ChatGPT alternative that operates fully offline on diverse hardware platforms, supporting universal architectures from PCs to multi-GPU clusters github | github profile

In 8 listsDetails