Awesome Data Analysis
Section: Tools · A storage layer that brings ACID transactions to Apache Spark and big data workloads.
FreshScore 80
Entry
Delta Lake
Appears in 6 awesome lists
Open-source storage framework enabling Lakehouse architecture with ACID transactions, scalable metadata handling, and unified batch/streaming processing. Apache 2.0 licensed.
Open github.comdelta-io/delta
Found in these lists
Awesome LLMOps
Section: Data Management · Storage layer that brings scalable, ACID transactions to Apache Spark and other engines.
ActiveScore 75
Awesome MLOps
Section: Data Management · Storage layer that brings scalable, ACID transactions to Apache Spark and other engines.
FreshScore 80
Awesome Open Source AI
Section: 1. Core Frameworks & Libraries · Open-source storage framework enabling Lakehouse architecture with ACID transactions, scalable metadata handling, and unified batch/streaming processing. Apache 2.0 licensed.
FreshScore 89
Awesome Production Machine Learning
Section: Data Storage Optimisation · Delta Lake is a storage layer that brings scalable, ACID transactions to Apache Spark and other big-data engines.
FreshScore 92
Awesome Spark
Section: Storage · Storage layer with ACID transactions.
SlowScore 66
Often listed next to it
Luigi
Python module for building complex pipelines of batch jobs. Handles dependency resolution, workflow management, visualization, and Hadoop integration. Built at Spotify and battle-tested in production. Apache 2.0 licensed.
In 15 listsDetails
Milvus
Milvus is a cloud-native, open-source vector database built to manage embedding vectors generated by machine learning models and neural networks.
In 14 listsDetails
Apache Airflow
"Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on DAGs a snap. The rich user interface makes it…
In 13 listsDetails
Dagster
Cloud-native orchestration platform for developing and maintaining data assets including ML models. Declarative programming model with integrated lineage and observability. Apache 2.0 licensed.
In 12 listsDetails
Prefect
Workflow management system that makes it easy to take your data pipelines and add semantics like retries, logging, dynamic mapping, caching, failure notifications, and more.
In 11 listsDetails
Dolt
Dolt is Git for Data! Dolt is a SQL database that you can fork, clone, branch, merge, push and pull just like a git repository.
In 10 listsDetails