Awesome Data Engineering
Section: Stream Processing · A streaming dataflow engine that provides data distribution, communication, and fault tolerance for distributed computations over data streams.
FreshScore 87
Entry
Apache Flink
Appears in 6 awesome lists
A streaming dataflow engine that provides data distribution, communication, and fault tolerance for distributed computations over data streams.
Found in these lists
AWESOME DATA SCIENCE
Section: Miscellaneous Tools · A platform for efficient, distributed, general-purpose data processing.
FreshScore 92
Awesome ETL
Section: Big Data (Hadoop Stack) · "a framework and distributed processing engine for stateful computations over unbounded and bounded data streams."
ActiveScore 72
Awesome Hadoop
Section: Realtime Data Processing · Apache Flink is a platform for efficient, distributed, general-purpose data processing. It supports exactly once stream processing.
StaleScore 48
Awesome Machine Learning
Section: Java · Open source platform for distributed stream and batch data processing.
FreshScore 93
Indie Hacker Tools Plus
Section: 缓存、消息队列与流处理 (Caching, MQ & Streaming)
FreshScore 86
Often listed next to it
Opik
Comet's open-source AI observability and evaluation platform: deep tracing of LLM calls, conversation logging, and agent activity, plus built-in eval metrics, prompt versioning, guardrails, and the Opik Agent Optimizer. Worth including because it unifies observability, verification, and…
In 16 listsDetails
Apache Airflow
"Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on DAGs a snap. The rich user interface makes it…
In 13 listsDetails
Ray
A fast and simple framework for building and running distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning library. ray.io
In 13 listsDetails
Gradio
Build and share delightful machine learning apps, all in Python. The de facto standard for creating interactive ML demos with automatic UI generation from function signatures. Powers thousands of Hugging Face Spaces.
In 11 listsDetails
Prefect
Workflow management system that makes it easy to take your data pipelines and add semantics like retries, logging, dynamic mapping, caching, failure notifications, and more.
In 11 listsDetails
MindsDB
MindsDB is an Explainable AutoML framework for developers. With MindsDB you can build, train and use state of the art ML models in as simple as one line of code.
In 11 listsDetails