Awesome local LLM
Section: Inference engines · Google's production-ready, high-performance, open-source inference framework for deploying Large Language Models on edge devices
FreshScore 87
Entry
LiteRT-LM
Appears in 4 awesome lists
LiteRT-LM is Google's production-ready, high-performance inference framework for deploying Large Language Models on edge devices, with cross-platform support for Android, iOS, Web, Desktop, and IoT.
Open github.comgoogle-ai-edge/litert-lm
Found in these lists
Awesome Open Source AI
Section: 3. Inference Engines & Serving · Production-ready runtime for deploying LLMs on edge devices with low-latency inference and optimized hardware paths for mobile and embedded platforms.
FreshScore 89
Awesome Production Machine Learning
Section: Deployment and Serving · LiteRT-LM is Google's production-ready, high-performance inference framework for deploying Large Language Models on edge devices, with cross-platform support for Android, iOS, Web, Desktop, and IoT.
FreshScore 92
awesome-cpp
Section: Other · LiteRT-LM is Google's production-ready, high-performance, open-source inference framework for deploying Large Language Models on edge devices.
FreshScore 79
Often listed next to it
LiteLLM
Unified proxy and SDK that routes to 100+ LLM providers behind a single OpenAI-compatible interface, with a Router handling retry/fallback across deployments, per-project cost and rate-limit tracking, and OTEL callback integrations. The right infrastructure layer when your harness needs provider…
In 16 listsDetails
LocalAI
robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video,…
In 14 listsDetails
Ollama
Ollama is a tool for running large language models locally, offering easy setup for macOS, Windows, Linux, and Docker, along with a library of models and quickstart guides for customization and integration github | github profile
In 12 listsDetails
vLLM
State-of-the-art serving engine with PagedAttention and continuous batching. Currently the fastest production-grade LLM server.
In 11 listsDetails
MindsDB
MindsDB is an Explainable AutoML framework for developers. With MindsDB you can build, train and use state of the art ML models in as simple as one line of code.
In 11 listsDetails
Open WebUI
Open WebUI is an extensible, feature-rich, and user-friendly self-hosted AI platform designed to operate entirely offline. It supports various LLM runners like Ollama and OpenAI-compatible APIs, with built-in inference engine for RAG, making it a powerful AI deployment solution.
In 11 listsDetails