Skip to content

Entry

ingestr

Appears in 6 awesome lists

CLI tool to copy data from any source to any destination with a single command. Load data into your warehouse before dbt transforms it. Supports 50+ sources including Postgres, MongoDB, Salesforce, Shopify.

Open github.combruin-data/ingestr

Found in these lists

Awesome Apache Airflow

Section: Libraries, Hooks, Utilities · CLI tool to copy data between any source and destination with a single command. Use with BashOperator to load data from 50+ sources (Postgres, MongoDB, Salesforce, etc.) into your warehouse.

FreshScore 82

Awesome Big Data

Section: Data Ingestion · CLI tool for copying data between sources and destinations.

ActiveScore 84

Awesome Data Engineering

Section: Data Ingestion · CLI tool to copy data between databases with a single command. Supports 50+ sources including PostgreSQL, MySQL, MongoDB, Salesforce, Shopify to any data warehouse.

FreshScore 87

Awesome dbt

Section: Integrations · CLI tool to copy data from any source to any destination with a single command. Load data into your warehouse before dbt transforms it. Supports 50+ sources including Postgres, MongoDB, Salesforce, Shopify.

FreshScore 82

Awesome DuckDB

Section: Integrations · CLI tool to copy data between databases and SaaS sources, with DuckDB supported as both a source and a destination.

FreshScore 87

Awesome ETL

Section: Libraries · "a CLI tool to copy data between any databases with a single command seamlessly."

ActiveScore 72

Logstash

Ship logs from any source, parse them, get the right timestamp, index them, and search them. Logstash is a tool for managing events and logs. You can use it to collect logs, parse them, and store them for later use (like, for searching). Speaking of searching, Logstash comes with a web interface…

In 8 listsDetails

Apache Pulsar

a distributed pub-sub messaging platform with a very flexible messaging model and an intuitive client API.

In 6 listsDetails

Bruin

Run and schedule dbt-style SQL transformations without Airflow. Adds data ingestion (50+ sources) and built-in data quality to the transformation layer. Open-source CLI or managed Bruin Cloud for teams who want dbt Cloud-like experience with ingestion included.

In 6 listsDetails

Kafka

🛠 - 🐙 - Distributed, fault tolerant, high throughput pub-sub messaging system.

In 6 listsDetails

Fluentd

Set Up Once, Collect More. Fluentd is an open source data collector designed for processing data streams. 150+ plugins instantly enable you to store the data for Log Management, Big Data Analytics, etc

In 5 listsDetails

Census

The easiest way to sync your customer data from your cloud data warehouse to SaaS applications like Salesforce, Marketo, HubSpot, Zendesk, etc. Census is the operational analytics platform that syncs your data warehouse with all your favorite apps. Get your customer success, sales & marketing…

In 5 listsDetails

Apache NiFi

Apache NiFi is an integrated data logistics platform for automating the movement of data between disparate systems.

In 5 listsDetails

dlt

data load tool is an open-source Python library for building data pipelines with schema inference and incremental loading. Includes an Airflow helper to convert dlt pipelines into Airflow DAGs/TaskGroups.

In 4 listsDetails