Skip to content
49

awesome-huggingface

🤗 A list of wonderful open-source projects & applications integrated with Hugging Face libraries.

1.1k stars109 forks56 entriesLast push Apr 29, 2024 (2 years ago)License Apache-2.0

This page lists names, links and short descriptions. The original list on GitHub is the source and belongs to its authors.

Official Libraries

transformers

State-of-the-art natural language processing for Jax, PyTorch and TensorFlow.

In 14 listsDetails

datasets

The largest hub of ready-to-use NLP datasets for ML models with fast, easy-to-use and efficient data manipulation tools.

In 6 listsDetails

tokenizers

Fast state-of-the-Art tokenizers optimized for research and production.

In 6 listsDetails

knockknock

Get notified when your training ends with only two additional lines of code.

In 2 lists

accelerate

A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision.

In 6 listsDetails

autonlp

Train state-of-the-art natural language processing models and deploy them in a scalable environment automatically.

nn_pruning

Prune a model while finetuning or training.

huggingface_hub

Client library to download and publish models and other files on the huggingface.co hub.

In 3 lists

tune

A benchmark for comparing Transformer-based models.

Tutorials

Official Course

(from Hugging Face) - The official course series provided by 🤗 Hugging Face.

transformers-tutorials

(by @nielsrogge) - Tutorials for applying multiple models on real-world datasets.

In 3 lists

NLP Toolkits

AllenNLP

(from AI2) - An open-source NLP research library.

In 5 listsDetails

Graph4NLP

Enabling easy use of Graph Neural Networks for NLP.

Lightning Transformers

Transformers with PyTorch Lightning interface.

In 2 lists

Adapter Transformers

Extension to the Transformers library, integrating adapters into state-of-the-art language models.

Obsei

A low-code AI workflow automation tool and performs various NLP tasks in the workflow pipeline.

Trapper

(from OBSS) - State-of-the-art NLP through transformer models in a modular design and consistent APIs.

Flair

A very simple framework for state-of-the-art NLP.

In 5 listsDetails

Text Representation

Sentence Transformers

(from UKPLab) - Widely used encoders computing dense vector representations for sentences, paragraphs, and images.

In 5 listsDetails

WhiteningBERT

(from Microsoft) - An easy unsupervised sentence embedding approach with whitening.

SimCSE

(from Princeton) - State-of-the-art sentence embedding with contrastive learning.

DensePhrases

(from Princeton) - Learning dense representations of phrases at scale.

Inference Engines

TurboTransformers

(from Tencent) - An inference engine for transformers with fast C++ API.

FasterTransformer

(from Nvidia) - A script and recipe to run the highly optimized transformer-based encoder and decoder component on NVIDIA GPUs.

In 4 listsDetails

lightseq

(from ByteDance) - A high performance inference library for sequence processing and generation implemented in CUDA.

In 2 lists

FastSeq

(from Microsoft) - Efficient implementation of popular sequence models (e.g., Bart, ProphetNet) for text generation, summarization, translation tasks etc.

Model Scalability

Parallelformers

(from TUNiB) - A library for model parallel deployment.

OSLO

(from TUNiB) - A library that supports various features to help you train large-scale models.

Deepspeed

(from Microsoft) - Deepspeed-ZeRO - scales any model size with zero to no changes to the model. Integrated with HF Trainer.

In 7 listsDetails

fairscale

(from Facebook) - Implements ZeRO protocol as well. Integrated with HF Trainer.

ColossalAI

(from Hpcaitech) - A Unified Deep Learning System for Large-Scale Parallel Training (1D, 2D, 2.5D, 3D and sequence parallelism, and ZeRO protocol).

In 14 listsDetails

Model Compression/Acceleration

torchdistill

PyTorch-based modular, configuration-driven framework for knowledge distillation.

In 2 lists

TextBrewer

(from HFL) - State-of-the-art distillation methods to compress language models.

BERT-of-Theseus

(from Microsoft) - Compressing BERT by progressively replacing the components of the original BERT.

Adversarial Attack

TextAttack

(from UVa) - A Python framework for adversarial attacks, data augmentation, and model training in NLP.

In 6 listsDetails

TextFlint

(from Fudan) - A unified multilingual robustness evaluation toolkit for NLP.

OpenAttack

(from THU) - An open-source textual adversarial attack toolkit.

Style Transfer

Styleformer

A neural language style transfer framework to transfer text smoothly between styles.

ConSERT

A contrastive framework for self-supervised sentence representation transfer.

Sentiment Analysis

conv-emotion

Implementation of different architectures for emotion recognition in conversations.

Grammatical Error Correction

Gramformer

A framework for detecting, highlighting and correcting grammatical errors on natural language text.

Translation

dl-translate

A deep learning-based translation library based on HF Transformers.

In 3 lists

EasyNMT

(from UKPLab) - Easy-to-use, state-of-the-art translation library and Docker images based on HF Transformers.

Knowledge and Entity

PURE

(from Princeton) - Entity and relation extraction from text.

Speech

s3prl

A self-supervised speech pre-training and representation learning toolkit.

speechbrain

A PyTorch-based speech toolkit.

In 5 listsDetails

Multi-modality

ViLT

(from Kakao) - A vision-and-language transformer Without convolution or region supervision.

Reinforcement Learning

trl

Fine-tune transformers using Proximal Policy Optimization (PPO) to align with human preferences.

In 3 lists

Question Answering

Haystack

(from deepset) - End-to-end framework for developing and deploying question-answering systems in the wild.

In 5 listsDetails

Recommender Systems

Transformers4Rec

(from Nvidia) - A flexible and efficient library powered by Transformers for sequential and session-based recommendations.

Evaluation

Jury

(from OBSS) - Easy to use tool for evaluating NLP model outputs, spesifically for NLG (Natural Language Generation), offering various automated text-to-text metrics.

In 2 lists

Spotlight

Interactively explore your HF dataset with one line of code. Use model results (e.g. embeddings, predictions) to understand critical data segments and model failure modes.

In 2 lists

Jina Integration

Jina integration of Hugging Face Accelerated API.

ColBERT

(from Stanford) - A fast and accurate retrieval model, enabling scalable BERT-based search over large text collections in tens of milliseconds.

Cloud

Amazon SageMaker

Making it easier than ever to train Hugging Face Transformer models in Amazon SageMaker.

Hardware

Qualcomm

Collaboration on enabling Transformers in Snapdragon.

tune

A benchmark for comparing Transformer-based models.

See category
94

Table of Contents

hesreallyhim/awesome-claude-code

A hand-picked collection of the finest of resources for the most awesome of agents, Claude Code, the undisputed champion of coding companions, from the unstoppable team…

Fresh★ 55k202 entriesPushed today
94

Awesome Agent Skills

VoltAgent/awesome-agent-skills

A curated collection of 1000+ agent skills from official dev teams and the community, compatible with Claude Code, Codex, Gemini CLI, Cursor, and more.

Fresh★ 35k839 entriesPushed today
93

Awesome Machine Learning

josephmisiti/awesome-machine-learning

A curated list of awesome Machine Learning frameworks, libraries and software.

Fresh★ 74k1188 entriesPushed 7 days ago
92

Awesome Production Machine Learning

EthicalML/awesome-production-machine-learning

A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning

Fresh★ 21k519 entriesPushed 3 days ago
92

AWESOME DATA SCIENCE

academic/awesome-datascience

:memo: An awesome Data Science repository to learn and apply for real world problems.

Fresh★ 30k881 entriesPushed today
91

Static Analysis

analysis-tools-dev/static-analysis

⚙️ A curated list of static analysis (SAST) tools and linters for all programming languages, config files, build tools, and more. The focus is on tools which improve…

Fresh★ 15k528 entriesPushed 8 days ago