Awesome Data Analysis
Section: Tools · A framework for stateful computations over unbounded and bounded data streams (real-time stream processing).
FreshScore 80
Entry
Apache Flink
Appears in 7 awesome lists
Stream processing framework with powerful batch and streaming capabilities. High-throughput, low-latency runtime with exactly-once processing guarantees. Ideal for real-time AI inference pipelines and event-driven ML applications. Apache 2.0 licensed.
Open github.comapache/flink
Found in these lists
Awesome HBase
Section: Integrations · Flink-HBase connector.
ActiveScore 65
Awesome Integration
Section: Stream Processing · Distributed stream processing framework with stateful computations, exactly-once semantics, and event-time processing at scale.
FreshScore 82
Awesome Java
Section: Data Processing · Apache-2.0 🟢Fast, reliable, large-scale data processing engine.
FreshScore 93
Awesome Open Source AI
Section: 1. Core Frameworks & Libraries · Stream processing framework with powerful batch and streaming capabilities. High-throughput, low-latency runtime with exactly-once processing guarantees. Ideal for real-time AI inference pipelines and event-driven ML applications. Apache 2.0 licensed.
FreshScore 89
Awesome Production Machine Learning
Section: Data Stream Processing · Open source stream processing framework with powerful stream and batch processing capabilities.
FreshScore 92
Useful Java links
Section: 7. Big data · Apache Flink is an open source stream processing framework with powerful stream- and batch-processing capabilities. License: Apache 2 , .
ActiveScore 74
Often listed next to it
Luigi
Python module for building complex pipelines of batch jobs. Handles dependency resolution, workflow management, visualization, and Hadoop integration. Built at Spotify and battle-tested in production. Apache 2.0 licensed.
In 15 listsDetails
Apache Airflow
"Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on DAGs a snap. The rich user interface makes it…
In 13 listsDetails
Dagster
Cloud-native orchestration platform for developing and maintaining data assets including ML models. Declarative programming model with integrated lineage and observability. Apache 2.0 licensed.
In 12 listsDetails
Prefect
Workflow management system that makes it easy to take your data pipelines and add semantics like retries, logging, dynamic mapping, caching, failure notifications, and more.
In 11 listsDetails
Apache Spark
Unified analytics engine for large-scale data processing. In-memory cluster computing with high-level APIs in Python, Scala, Java, and R. Powers MLlib for distributed machine learning and Structured Streaming for real-time data. Apache 2.0 licensed.
In 8 listsDetails
Kestra
Event-driven orchestration and scheduling platform for mission-critical workflows. Infrastructure-as-Code approach with declarative YAML, Git version control integration, and hundreds of plugins for data pipelines and ML workflows. Apache 2.0 licensed.
In 8 listsDetails
Kedro
Toolbox for production-ready data science. Uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular. Apache 2.0 licensed.
In 7 listsDetails
dbt-core
Project banner, super clear description (friendly to people brand new to the product), screenshot of the docs the tool generates, and concise links to other comprehensive pages explaining Getting Started, Joining the dbt Community, Bug Reports, Code Contribution, and Code of Conduct
In 7 listsDetails