Awesome Data Engineering
Section: Data Lake Management · An open source platform that delivers resilience and manageability to object-storage based data lakes.
FreshScore 87
Entry
lakeFS
Appears in 10 awesome lists
Data version control for your data lake that transforms object storage into Git-like repositories. Enables atomic, versioned data lake operations with branching, committing, and merging for data pipelines. Apache 2.0 licensed.
Open github.comtreeverse/lakefs
Found in these lists
Awesome LLMOps
Section: Data Storage · Git-like capabilities for your object storage.
ActiveScore 75
Awesome MLOps
Section: Data Management · Repeatable, atomic and versioned data lake on top of object storage.
FreshScore 80
Awesome Open Source AI
Section: 1. Core Frameworks & Libraries · Data version control for your data lake that transforms object storage into Git-like repositories. Enables atomic, versioned data lake operations with branching, committing, and merging for data pipelines. Apache 2.0 licensed.
FreshScore 89
Awesome Opensource Data Engineering
Section: General · ] - Repeatable, atomic and versioned data lake on top of object storage.
StaleScore 51
Awesome Pipeline
Section: ETL & Data orchestration · Repeatable, atomic and versioned data lake on top of object storage.
FreshScore 86
Awesome Production Machine Learning
Section: Model, Data and Experiment Management · Repeatable, atomic and versioned data lake on top of object storage.
FreshScore 92
Awesome Storage
Section: S3 compatible file servers · lakeFS is an open source tool that transforms your object storage into a Git-like repository. It enables you to manage your data lake the way you manage your code.
StaleScore 50
go-awesome
Section: 文件/存储 · 类 Git 文件对象存储
StaleScore 56
awesome-go
Section: Data Science and Analytics · lakeFS - Data version control for your data lake | Git for data
FreshScore 81
Often listed next to it
Milvus
Milvus is a cloud-native, open-source vector database built to manage embedding vectors generated by machine learning models and neural networks.
In 14 listsDetails
Dolt
Dolt is Git for Data! Dolt is a SQL database that you can fork, clone, branch, merge, push and pull just like a git repository.
In 10 listsDetails
JuiceFS
A high-performance Cloud-Native file system driven by object storage for large-scale data storage.
In 9 listsDetails
Qdrant
Vector Search Engine and Database for the next generation of AI applications. Also available in the cloud
In 8 listsDetails
Pinecone
The Pinecone vector database makes it easy to build high-performance vector search applications. Developer-friendly, fully managed, and easily scalable without infrastructure hassles.
In 8 listsDetails
Delta Lake
Open-source storage framework enabling Lakehouse architecture with ACID transactions, scalable metadata handling, and unified batch/streaming processing. Apache 2.0 licensed.
In 6 listsDetails
Marquez
LF AI & Data Foundation Graduated project for metadata collection, aggregation, and visualization. Maintains provenance of how datasets are consumed and produced with global visibility into job runtime and dataset lifecycle management. Integrates with OpenLineage. Apache 2.0 licensed.
In 5 listsDetails