Awesome Big Data
Section: Distributed Programming · A fast and simple framework for building and running distributed applications.
Entry
Appears in 14 awesome lists
A fast and simple framework for building and running distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning library. ray.io
Section: Distributed Programming · A fast and simple framework for building and running distributed applications.
Section: Miscellaneous Tools · Scalable hyperparameter tuning library
Section: AutoML · Scalable Hyperparameter Tuning.
Section: Python · RLlib is an industry level, highly scalable RL library for tf and torch, based on Ray. It's used by companies like Amazon and Microsoft to solve real-world decision making problems at scale.
Section: Optimization Tools · Fast and simple framework for building and running distributed applications.
Section: 7. Training & Fine-tuning Ecosystem · Scalable distributed training.
Section: Pipeline frameworks & libraries · Flexible, high-performance distributed Python execution framework.
Section: Computation and Communication Optimisation · Ray is a flexible, high-performance distributed execution framework for machine learning.
Section: Concurrency and Parallelism · Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Section: Other libraries: · A fast and simple framework for building and running distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning library. ray.io
Section: Machine Learning · A fast and simple framework for building and running distributed applications.
Section: Graph Computation · An open source framework that provides a simple, universal API for building distributed applications.
Section: Distributed Computing · A unified framework for scaling AI and Python applications.
Section: Computation · | Python, C++ | - An open source framework that provides a simple, universal API for building distributed applications.
Comet's open-source AI observability and evaluation platform: deep tracing of LLM calls, conversation logging, and agent activity, plus built-in eval metrics, prompt versioning, guardrails, and the Opik Agent Optimizer. Worth including because it unifies observability, verification, and…
"Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on DAGs a snap. The rich user interface makes it…
industrial-grade ASR toolkit; 170× realtime on GPU, 50+ languages, built-in VAD, punctuation, speaker diarization, and emotion detection. Includes non-autoregressive SenseVoice and LLM-based Fun-ASR-Nano models.
Build and share delightful machine learning apps, all in Python. The de facto standard for creating interactive ML demos with automatic UI generation from function signatures. Powers thousands of Hugging Face Spaces.
The fastest way to build and share data apps. Transform Python scripts into beautiful web applications with minimal code. Widely used for ML model demos, data visualization, and internal tools.
Workflow management system that makes it easy to take your data pipelines and add semantics like retries, logging, dynamic mapping, caching, failure notifications, and more.