Skip to content

Entry

Hamilton

Appears in 9 awesome lists

| Python | - A scalable general purpose micro-framework for defining dataflows. You can use it to build dataframes, numpy matrices, python objects, ML models, etc. Embed Hamilton anywhere python runs, e.g. spark, airflow, jupyter, fastapi, python scripts, etc.

Open github.comdagworks-inc/hamilton

Found in these lists

Awesome Data & AI Governance

Section: Hamilton

SlowScore 57

Awesome Data Engineering

Section: Workflow · A lightweight library to define data transformations as a directed-acyclic graph (DAG). If you like dbt for SQL transforms, you will like Hamilton for Python processing.

FreshScore 93

Awesome ETL

Section: Libraries · "helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does."

SlowScore 74

Awesome LLMOps

Section: Workflow · A lightweight framework to represent ML/language model pipelines as a series of python functions.

FreshScore 91

Awesome Machine Learning

Section: Tools · a lightweight library to define data transformations as a directed-acyclic graph (DAG). It helps author reliable feature engineering and machine learning pipelines, and more.

FreshScore 93

Awesome MLOps

Section: Workflow Tools · A scalable general purpose micro-framework for defining dataflows.

FreshScore 86

Awesome Pipeline

Section: Pipeline frameworks & libraries · A python micro-framework for describing dataflows; runs anywhere python runs.

FreshScore 82

Awesome Python Data Science

Section: Pipelines · A microframework for dataframe generation that applies Directed Acyclic Graphs specified by a flow of lazily evaluated Python functions.

SlowScore 70

Awesome Systematic Trading

Section: Computation · | Python | - A scalable general purpose micro-framework for defining dataflows. You can use it to build dataframes, numpy matrices, python objects, ML models, etc. Embed Hamilton anywhere python runs, e.g. spark, airflow, jupyter, fastapi, python scripts, etc.

FreshScore 85

Luigi

Python module for building complex pipelines of batch jobs. Handles dependency resolution, workflow management, visualization, and Hadoop integration. Built at Spotify and battle-tested in production. Apache 2.0 licensed.

In 15 listsDetails

Apache Airflow

"Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on DAGs a snap. The rich user interface makes it…

In 13 listsDetails

Dagster

Cloud-native orchestration platform for developing and maintaining data assets including ML models. Declarative programming model with integrated lineage and observability. Apache 2.0 licensed.

In 12 listsDetails

Dask

| Python | - Parallel computing with task scheduling in Python with a Pandas like API

In 11 listsDetails

Polars

| Rust, Python | - Polars is a blazingly fast DataFrames library implemented in Rust using Apache Arrow Columnar Format as memory model.

In 10 listsDetails

DuckDB

A fast in-process analytical database that has zero external dependencies, runs on Linux/macOS/Windows, offers a rich SQL dialect, and is free and extensible.

In 8 listsDetails

Kestra

Event-driven orchestration and scheduling platform for mission-critical workflows. Infrastructure-as-Code approach with declarative YAML, Git version control integration, and hundreds of plugins for data pipelines and ML workflows. Apache 2.0 licensed.

In 8 listsDetails

xmltodict

"Python module that makes working with XML feel like you are working with JSON."

In 5 listsDetails