Skip to content

Entry

LocalAI

Appears in 4 awesome lists

is a self-hosted, community-driven, local OpenAI-compatible API. Drop-in replacement for OpenAI running LLMs on consumer-grade hardware with no GPU required. It's an API to run ggml compatible models: llama, gpt4all, rwkv, whisper, vicuna, koala, gpt4all-j, cerebras, falcon, dolly, starcoder, and…

Open localai.io

Found in these lists

Awesome AI Coding Tools

Section: Local LLM Tools · Open-source OpenAI-compatible API for running models locally without GPU.

FreshScore 82

Awesome-Selfhosted

Section: Generative Artificial Intelligence (GenAI) · Run your AI models locally and generate images and audio (alternative to OpenAI and Claude). (Source Code, Clients) MIT Docker/K8S

FreshScore 98

Awesome Privacy

Section: Artificial Intelligence · Self-hosted OpenAI-compatible API that runs language, image and voice models on your own hardware. self-host · MIT

FreshScore 98

Table of Contents

Section: LLMs · is a self-hosted, community-driven, local OpenAI-compatible API. Drop-in replacement for OpenAI running LLMs on consumer-grade hardware with no GPU required. It's an API to run ggml compatible models: llama, gpt4all, rwkv, whisper, vicuna, koala, gpt4all-j, cerebras, falcon, dolly, starcoder, and…

SlowScore 60

Whisper

General-purpose speech recognition model for multilingual transcription, translation, language identification, and related speech tasks. Python · Self-host · infra.

In 13 listsDetails

FunASR

industrial-grade ASR toolkit; 170× realtime on GPU, 50+ languages, built-in VAD, punctuation, speaker diarization, and emotion detection. Includes non-autoregressive SenseVoice and LLM-based Fun-ASR-Nano models.

In 12 listsDetails

llama.cpp

Pure C/C++ inference engine with GGUF format support. The gold standard for CPU/GPU/Apple Silicon on-device running. Includes llama-server for OpenAI-compatible API. Now at 100K+ stars.

In 12 listsDetails

Aider

Terminal pair programmer that edits code in your local git repository with the model you choose. CLI · Apache-2.0

In 9 listsDetails

Ollama

Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, and other large language models. (Source Code) MIT Docker/Python

In 9 listsDetails

LM Studio

LM Studio offers a platform for running various local LLMs like LLaMa, Falcon, MPT, and others offline, featuring a Chat UI, OpenAI-compatible server, and model downloads from Hugging Face, with support for Mac, Windows, and Linux, emphasizing privacy and no data collection, free for personal use…

In 11 listsDetails

shimmy

Python-free Rust inference server with OpenAI API compatibility. Supports GGUF and SafeTensors formats with hot model swap, auto-discovery, and single binary deployment for zero-dependency inference. Apache 2.0 licensed.

In 7 listsDetails

Local Deep Research

Local Deep Research achieves ~95% on SimpleQA benchmark (tested with GPT-4.1-mini). Supports local and cloud LLMs (Ollama, Google, Anthropic, ...). Searches 10+ sources - arXiv, PubMed, web, and your private documents. Everything Local.

In 7 listsDetails