awesome-ChatGPT-repositories
Section: NLP · Open-source end-to-end LLM Development Platform
FreshScore 87
Entry
Opik
Appears in 16 awesome lists
Comet's open-source AI observability and evaluation platform: deep tracing of LLM calls, conversation logging, and agent activity, plus built-in eval metrics, prompt versioning, guardrails, and the Opik Agent Optimizer. Worth including because it unifies observability, verification, and…
Open github.comcomet-ml/opik
Found in these lists
Awesome Data Analysis
Section: Tools · ML platform for tracking, comparing and optimizing experiments.
FreshScore 80
AWESOME DATA SCIENCE
Section: Miscellaneous Tools · Evaluate, test, and ship LLM applications across your dev and production lifecycles.
FreshScore 92
Awesome Generative AI
Section: Large Language Models (LLMs) · Evaluate, test, and ship LLM applications with a suite of observability tools to calibrate language model outputs across your dev and production lifecycle.
SlowScore 62
Awesome Generative AI Data Scientist
Section: Testing and Monitoring (Observability) · Opik is an open-source platform for evaluating, testing, and monitoring LLM applications.
SlowScore 52
Awesome GPT Prompt Engineering
Section: Playgrounds and Alternative UIs · Evaluate, test, and ship LLM applications across your dev and production lifecycles.
SlowScore 65
Awesome Harness Engineering
Section: Observability & Tracing · Comet's open-source AI observability and evaluation platform: deep tracing of LLM calls, conversation logging, and agent activity, plus built-in eval metrics, prompt versioning, guardrails, and the Opik Agent Optimizer. Worth including because it unifies observability, verification, and…
FreshScore 88
Awesome LLMOps
Section: LLMOps · Confidently evaluate, test, and ship LLM applications with a suite of observability tools to calibrate language model outputs across your dev and production lifecycle.
ActiveScore 75
Awesome local LLM
Section: Testing, Evaluation and Observability · debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards
FreshScore 87
Awesome Machine Learning
Section: Python · Evaluate, trace, test, and ship LLM applications across your dev and production lifecycles.
FreshScore 93
Awesome MLOps
Section: Visual Analysis and Debugging · Evaluate, test, and ship LLM applications with a suite of observability tools.
FreshScore 80
Awesome Open Source AI
Section: 8. MLOps / LLMOps & Production · Production-ready LLM evaluation platform.
FreshScore 89
Awesome Production Machine Learning
Section: Evaluation and Monitoring · Opik is an open-source platform for evaluating, testing and monitoring LLM applications.
FreshScore 92
Awesome Chatgpt
Section: Hosted and self-hosted · Open-source end-to-end LLM development platform.
SlowScore 69
Awesome Generative AI
Section: Developer tools · An open-source platform for tracing, evaluating, and monitoring LLM applications. #opensource
FreshScore 90
awesome-python
Section: Other · Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
FreshScore 81
Often listed next to it
LiteLLM
Unified proxy and SDK that routes to 100+ LLM providers behind a single OpenAI-compatible interface, with a Router handling retry/fallback across deployments, per-project cost and rate-limit tracking, and OTEL callback integrations. The right infrastructure layer when your harness needs provider…
In 16 listsDetails
transformers
(formerly known as pytorch-transformers and pytorch-pretrained-bert) provides state-of-the-art general-purpose architectures (BERT, GPT-2, RoBERTa, XLM, DistilBert, XLNet, CTRL...) for Natural Language Understanding (NLU) and Natural Language Generation (NLG) with over 32+ pretrained models in…
In 14 listsDetails
Colossal-AI
(from Hpcaitech) - A Unified Deep Learning System for Large-Scale Parallel Training (1D, 2D, 2.5D, 3D and sequence parallelism, and ZeRO protocol).
In 14 listsDetails
Apache Airflow
"Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on DAGs a snap. The rich user interface makes it…
In 13 listsDetails
Mem0
Mem0 is an intelligent memory layer for Large Language Models that enhances personalized AI experiences by retaining and utilizing contextual information across various applications. github | website | docs | discord | twitter | github profile | linkedin
In 13 listsDetails
Ray
A fast and simple framework for building and running distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning library. ray.io
In 13 listsDetails
Haystack
Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search,…
In 13 listsDetails