Awesome local LLM
Section: Inference engines · Google's on-device framework for high-performance ML & GenAI deployment on edge platforms, via efficient conversion, runtime, and optimization
FreshScore 87
Entry
LiteRT
Appears in 4 awesome lists
Google's production-ready on-device ML and GenAI deployment framework. Supports Android, iOS, Web, Desktop, and IoT targets with GPU/NPU acceleration via a unified edge-first runtime. Apache 2.0 licensed.
Open github.comgoogle-ai-edge/litert
Found in these lists
Awesome Open Source AI
Section: 3. Inference Engines & Serving · Google's production-ready on-device ML and GenAI deployment framework. Supports Android, iOS, Web, Desktop, and IoT targets with GPU/NPU acceleration via a unified edge-first runtime. Apache 2.0 licensed.
FreshScore 89
Awesome Production Machine Learning
Section: Deployment and Serving · LiteRT (formerly TensorFlow Lite) is Google's high-performance runtime for on-device AI inference, enabling deployment of machine learning models on mobile, embedded, and edge devices.
FreshScore 92
awesome-cpp
Section: Other · LiteRT, successor to TensorFlow Lite. is Google's On-device framework for high-performance ML & GenAI deployment on edge platforms, via efficient conversion, runtime, and optimization
FreshScore 79
Often listed next to it
LiteLLM
Unified proxy and SDK that routes to 100+ LLM providers behind a single OpenAI-compatible interface, with a Router handling retry/fallback across deployments, per-project cost and rate-limit tracking, and OTEL callback integrations. The right infrastructure layer when your harness needs provider…
In 16 listsDetails
LocalAI
robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video,…
In 14 listsDetails
Ollama
Ollama is a tool for running large language models locally, offering easy setup for macOS, Windows, Linux, and Docker, along with a library of models and quickstart guides for customization and integration github | github profile
In 12 listsDetails
vLLM
State-of-the-art serving engine with PagedAttention and continuous batching. Currently the fastest production-grade LLM server.
In 11 listsDetails
MindsDB
MindsDB is an Explainable AutoML framework for developers. With MindsDB you can build, train and use state of the art ML models in as simple as one line of code.
In 11 listsDetails
Open WebUI
Open WebUI is an extensible, feature-rich, and user-friendly self-hosted AI platform designed to operate entirely offline. It supports various LLM runners like Ollama and OpenAI-compatible APIs, with built-in inference engine for RAG, making it a powerful AI deployment solution.
In 11 listsDetails