Skip to content

Entry

Opik

Appears in 18 awesome lists

Comet's open-source AI observability and evaluation platform: deep tracing of LLM calls, conversation logging, and agent activity, plus built-in eval metrics, prompt versioning, guardrails, and the Opik Agent Optimizer. Worth including because it unifies observability, verification, and…

Open github.comcomet-ml/opik

Found in these lists

awesome-ChatGPT-repositories

Section: NLP · Open-source end-to-end LLM Development Platform

FreshScore 87

Awesome Data Analysis

Section: Tools · ML platform for tracking, comparing and optimizing experiments.

FreshScore 79

AWESOME DATA SCIENCE

Section: Miscellaneous Tools · Evaluate, test, and ship LLM applications across your dev and production lifecycles.

FreshScore 93

Awesome Dataviz

Section: Machine Learning tools · Formerly CometLLM. Debug, evaluate, and monitor LLM applications with tracing and dashboards.

FreshScore 92

Awesome Agent Evals

Section: 5f · Observability + eval platforms (tracing · datasets · online/offline · CI) · Comet — https://github.com/comet-ml/opik — 🆕 fully-OSS eval + observability (judges, datasets, CI-runnable evals).

FreshScore 87

Awesome Generative AI

Section: Large Language Models (LLMs) · Evaluate, test, and ship LLM applications with a suite of observability tools to calibrate language model outputs across your dev and production lifecycle.

SlowScore 58

Awesome Generative AI Data Scientist

Section: Testing and Monitoring (Observability) · Opik is an open-source platform for evaluating, testing, and monitoring LLM applications.

StaleScore 54

Awesome GPT Prompt Engineering

Section: Playgrounds and Alternative UIs · Evaluate, test, and ship LLM applications across your dev and production lifecycles.

SlowScore 62

Awesome Harness Engineering

Section: Observability & Tracing · Comet's open-source AI observability and evaluation platform: deep tracing of LLM calls, conversation logging, and agent activity, plus built-in eval metrics, prompt versioning, guardrails, and the Opik Agent Optimizer. Worth including because it unifies observability, verification, and…

FreshScore 93

Awesome LLMOps

Section: LLMOps · Confidently evaluate, test, and ship LLM applications with a suite of observability tools to calibrate language model outputs across your dev and production lifecycle.

FreshScore 91

Awesome local LLM

Section: Testing, Evaluation and Observability · debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards

FreshScore 90

Awesome Machine Learning

Section: Python · Evaluate, trace, test, and ship LLM applications across your dev and production lifecycles.

FreshScore 93

Awesome MLOps

Section: Visual Analysis and Debugging · Evaluate, test, and ship LLM applications with a suite of observability tools.

FreshScore 86

Awesome Open Source AI

Section: 8. MLOps / LLMOps & Production · Production-ready LLM evaluation platform.

FreshScore 87

Awesome Production Machine Learning

Section: Evaluation and Monitoring · Opik is an open-source platform for evaluating, testing and monitoring LLM applications.

FreshScore 98

awesome-python

Section: Other · Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

FreshScore 79

Awesome Chatgpt

Section: Hosted and self-hosted · Open-source end-to-end LLM development platform.

FreshScore 89

Awesome Generative AI

Section: Developer tools · An open-source platform for tracing, evaluating, and monitoring LLM applications. #opensource

FreshScore 90

LiteLLM

Unified proxy and SDK that routes to 100+ LLM providers behind a single OpenAI-compatible interface, with a Router handling retry/fallback across deployments, per-project cost and rate-limit tracking, and OTEL callback integrations. The right infrastructure layer when your harness needs provider…

In 16 listsDetails

transformers

(formerly known as pytorch-transformers and pytorch-pretrained-bert) provides state-of-the-art general-purpose architectures (BERT, GPT-2, RoBERTa, XLM, DistilBert, XLNet, CTRL...) for Natural Language Understanding (NLU) and Natural Language Generation (NLG) with over 32+ pretrained models in…

In 14 listsDetails

mem0

The most-deployed memory layer for AI agents — drop-in add/search/update/delete fact storage with LLM-extracted long-term / short-term / semantic memory: automatic fact consolidation (dedup, contradiction resolution, decay) so agents keep "context that persists" without context bloat;…

In 14 listsDetails

Ray

A fast and simple framework for building and running distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning library. ray.io

In 14 listsDetails

Colossal-AI

(from Hpcaitech) - A Unified Deep Learning System for Large-Scale Parallel Training (1D, 2D, 2.5D, 3D and sequence parallelism, and ZeRO protocol).

In 14 listsDetails

Apache Airflow

"Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on DAGs a snap. The rich user interface makes it…

In 13 listsDetails

Haystack

Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search,…

In 13 listsDetails

FunASR

industrial-grade ASR toolkit; 170× realtime on GPU, 50+ languages, built-in VAD, punctuation, speaker diarization, and emotion detection. Includes non-autoregressive SenseVoice and LLM-based Fun-ASR-Nano models.

In 12 listsDetails