Skip to content

Entry

DataOps TestGen

Appears in 4 awesome lists

Open-source data quality tool that profiles tables and generates and runs data quality tests; connects to DuckDB databases and uses DuckDB to profile and test CSV, Parquet, and JSON files locally, in Amazon S3, or in Azure Blob Storage. Apache 2.0.

Open github.comdatakitchen/dataops-testgen

Found in these lists

Awesome Big Data

Section: Data Quality and Observability · open-source data quality tool that profiles tables, generates tests from the profiling results, and monitors freshness, volume, and schema, with a web UI.

FreshScore 90

Awesome Data Engineering

Section: Testing · Open-source data quality tool that profiles tables, generates validation tests from the profiling results, and monitors freshness, volume, and schema from a web UI.

FreshScore 93

Awesome DuckDB

Section: Data Quality & Governance · Open-source data quality tool that profiles tables and generates and runs data quality tests; connects to DuckDB databases and uses DuckDB to profile and test CSV, Parquet, and JSON files locally, in Amazon S3, or in Azure Blob Storage. Apache 2.0.

FreshScore 91

Awesome MLOps

Section: Data Validation · Profiles tables, generates data quality tests, and monitors freshness, volume, and schema.

FreshScore 86

Dex

Gemini CLI extension with three Agent Skills for analytics engineering: explore a warehouse or DuckDB file read-only (profiles, PII flags, verified joins), author dbt models and semantic layers as reviewable diffs, and detect schema drift. Install with gemini extensions install…

In 11 listsDetails

Great Expectations

Open Source data validation framework to manage data quality. Users can define and document “expectations” rules about how data should look and behave.

In 4 listsDetails

Grai

A data catalog tool that integrates into your CI system exposing downstream impact testing of data changes. These tests prevent data changes which might break data pipelines or BI dashboards from making it to production.

In 3 lists

Data Contract CLI

Command-line tool to test data against data contracts in the Open Data Contract Standard (ODCS). Converts from and to formats such as SQL DDL, dbt, Avro, and JSON Schema. Uses DuckDB to run schema and quality checks on sources such as CSV, Parquet and JSON files (local, S3, GCS, Azure), DuckDB…

In 3 lists

Scherlok

Zero-config data quality CLI. Profiles every table on first run, then auto-detects anomalies (volume drops, schema drift, freshness misses, distribution shifts) on subsequent runs. No YAML, no rules to write. Works with Postgres, BigQuery, Snowflake, and dbt.

In 3 lists

DataDriven

Interview practice with SQL query execution, Python, and data modeling exercises.

In 2 lists

dbmask

Open-source tool that scans SQL databases for sensitive columns, masks them with deterministic fakes, and validates the masked copy row by row against the original.

In 2 lists

Snowflake Emulator

A lightweight Snowflake emulator built with Go and DuckDB for local development and testing.

In 2 lists