Awesome Apache Airflow
Section: Vital links · (also the official website)
Entry
Appears in 9 awesome lists
is an open-source workflow management platform created by the community to programmatically author, schedule and monitor workflows. Install. Principles. Scalable. Airflow has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers. Airflow is ready to scale to…
Section: Vital links · (also the official website)
Section: Software · A platform to programmatically author, schedule, and monitor workflows.
Section: Workflow · A platform to programmatically author, schedule and monitor workflows.
Section: Data Processing · Platform to programmatically author, schedule, and monitor workflows.
Section: Scalability
Section: Automation · Platform to programmatically author, schedule, and monitor workflows. (Source Code) Apache-2.0 Python/Docker
Section: Tooling · Programmatically author, schedule and monitor workflows.
Section: ML frameworks & applications · is an open-source workflow management platform created by the community to programmatically author, schedule and monitor workflows. Install. Principles. Scalable. Airflow has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers. Airflow is ready to scale to…
Section: ML Frameworks, Libraries, and Tools · is an open-source workflow management platform created by the community to programmatically author, schedule and monitor workflows. Install. Principles. Scalable. Airflow has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers. Airflow is ready to scale to…
is an open platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly working in collaboration with cloud, Linux, and Windows vendors, including Microsoft.
is a free software application used for event monitoring and alerting. It records real-time metrics in a time series database (allowing for high dimensionality) built using a HTTP pull model, with flexible queries and real-time alerting.
"Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on DAGs a snap. The rich user interface makes it…
Workflow management system that makes it easy to take your data pipelines and add semantics like retries, logging, dynamic mapping, caching, failure notifications, and more.
is an open-source system for automating deployment, scaling, and management of containerized applications.
Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo Workflows is implemented as a Kubernetes CRD (Custom Resource Definition).