Awesome Big Data
Section: Data Ingestion · Open-source replication of Postgres, MySQL, MongoDB, Oracle, SQL Server, DB2, Kafka and S3 into Apache Iceberg, with CDC and table maintenance. Written in Go.
Entry
Appears in 4 awesome lists
Open-source replication of Postgres, MySQL, MongoDB, Oracle, SQL Server, DB2, Kafka and S3 into Apache Iceberg, with CDC and table maintenance. Written in Go.
Section: Data Ingestion · Open-source replication of Postgres, MySQL, MongoDB, Oracle, SQL Server, DB2, Kafka and S3 into Apache Iceberg, with CDC and table maintenance. Written in Go.
Section: Data Ingestion · Open-source replication of Postgres, MySQL, MongoDB, Oracle, SQL Server, DB2, Kafka and S3 into Apache Iceberg, with CDC and table maintenance. Written in Go.
Section: Data · Open-source MongoDB to Apache Iceberg replication using change streams or oplog CDC, with parallel full loads. Written in Go
Section: Extract, transform, load (ETL) · Open-source replication of Postgres, MySQL, MongoDB, Oracle, SQL Server, DB2, Kafka and S3 into Apache Iceberg, with CDC and table maintenance. Written in Go.
is a tool for managing events and logs. When used generically, the term encompasses a larger system of log collection, processing, storage and searching activities.
Log collector and forwarder, not a SIEM. Gathers logs from many sources and routes them to other systems for processing, integrating cleanly with Elasticsearch, OpenSearch, Splunk, and Snowflake. Does not perform threat detection, correlation, or alerting, and has no storage layer. Highly…
🛠 - 🐙 - Distributed, fault tolerant, high throughput pub-sub messaging system.
CLI tool to copy data from any source to any destination with a single command. Load data into your warehouse before dbt transforms it. Supports 50+ sources including PostgreSQL, MongoDB, Salesforce, Shopify.
Run and schedule dbt-style SQL transformations without Airflow. Adds data ingestion (50+ sources) and built-in data quality to the transformation layer. Open-source CLI or managed Bruin Cloud for teams who want dbt Cloud-like experience with ingestion included.
a distributed pub-sub messaging platform with a very flexible messaging model and an intuitive client API.
The easiest way to sync your customer data from your cloud data warehouse to SaaS applications like Salesforce, Marketo, HubSpot, Zendesk, etc. Census is the operational analytics platform that syncs your data warehouse with all your favorite apps. Get your customer success, sales & marketing…
Apache NiFi is an integrated data logistics platform for automating the movement of data between disparate systems.