awesome-ChatGPT-repositories
Section: Langchain · Python bindings for llama.cpp
Entry
Appears in 5 awesome lists
demonstrates integration, including setup using the Instructor library, with examples of retrieving weather information and extracting user details.
Section: Langchain · Python bindings for llama.cpp
Section: Jupyter Notebooks · demonstrates integration, including setup using the Instructor library, with examples of retrieving weather information and extracting user details.
Section: 3. Inference Engines & Serving · Official Python bindings for llama.cpp.
Section: LLM and Inference · Python bindings for llama.cpp
Section: AI and Agents · Python bindings for llama.cpp, for running GGUF models locally.
Langchain integrates various providers like Anthropic, AWS, and OpenAI, and offers tools for components such as LLMs, chat models, and data analysis, supporting functionalities from Alpha Vantage to YouTube github | docs
Unified proxy and SDK that routes to 100+ LLM providers behind a single OpenAI-compatible interface, with a Router handling retry/fallback across deployments, per-project cost and rate-limit tracking, and OTEL callback integrations. The right infrastructure layer when your harness needs provider…
(MIT) provides modules for structured outputs at different levels of abstraction, including output parsers for text completion endpoints, Pydantic programs for mapping prompts to structured outputs using function calling or output parsing, and pre-defined Pydantic programs for specific output types.
Ollama is a tool for running large language models locally, offering easy setup for macOS, Windows, Linux, and Docker, along with a library of models and quickstart guides for customization and integration github | github profile
Open WebUI is an extensible, feature-rich, and user-friendly self-hosted AI platform designed to operate entirely offline. It supports various LLM runners like Ollama and OpenAI-compatible APIs, with built-in inference engine for RAG, making it a powerful AI deployment solution.
Pure C/C++ inference engine with GGUF format support. The gold standard for CPU/GPU/Apple Silicon on-device running. Includes llama-server for OpenAI-compatible API. Now at 100K+ stars.
State-of-the-art serving engine with PagedAttention and continuous batching. Currently the fastest production-grade LLM server.
🟢🟠 — Apache-licensed AI and MCP gateway with multi-provider routing, virtual-key access controls, budgets, rate limits, MCP aggregation, OAuth, automatic fallbacks, and load balancing. (Maxim) — note: model-provider credentials and proxied request/response data are sensitive; restrict and…;…