Skip to content
85

Awesome Model Quantization

A curated collection of papers, benchmarks, surveys, and tools for model quantization, covering low-bit networks, LLMs, multimodal and generative models, vector and lattice quantization, and efficient deployment.

2.5k stars246 forks35 entriesLast push Nov 16, 2026 (today)License none

This page lists names, links and short descriptions. The original list on GitHub is the source and belongs to its authors.

Benchmarks

MQBench: Towards Reproducible and Deployable Model Quantization Benchmark

QAT + deploymentCompares quantization algorithms under reproducible settings and hardware backend constraints.

BiBench: Benchmarking and Analyzing Network Binarization

Binary networksCompares binarization methods across tasks, architectures and deployment settings.

Evaluating Quantized Large Language Models

Weights, activations + KV cacheEvaluates 11 model families on basic NLP, emergent abilities, trustworthiness, dialogue and long-context tasks.

LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit

LLM toolkitCompares calibration data, method pipelines and quantization configurations; the toolkit is now LightCompress.

An empirical study of LLaMA3 quantization: from LLMs to MLLMs

LLMs + multimodalExamines low-bit behavior across LLaMA3 language and multimodal models.

An Empirical Study of Qwen3 Quantization

Dense + MoE LLMsStudies quantization across Qwen3 model sizes, architectures and reasoning settings.

RobustMQ: Benchmarking Robustness of Quantized Models

Model robustnessTests quantized models beyond clean accuracy, including robustness under input perturbations.

Qwen3 study

Preprint

Survey Papers

A White Paper on Neural Network Quantization

Practical PTQ + QATExplains quantizer design, common failure modes and practical post-training and quantization-aware training workflows.

A Survey of Quantization Methods for Efficient Neural Network Inference

Foundations + taxonomyReviews quantization design choices, mixed precision and the trade-offs between model accuracy and efficient inference.

Binary Neural Networks: A Survey

Binary networksSurveys binary network representations, training methods and applications.

Link may be dead

A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithms

LLM algorithms + systemsConnects low-bit LLM algorithms with numerical formats and inference systems.

Link may be dead

Low-bit Model Quantization for Deep Neural Networks: A Survey

Broad low-bit methodsMaps low-bit quantization methods across neural network architectures and applications.

Quantization methods survey

Chapter

Binary networks survey

Blog

Books

Quantization and Fast Inference: A practitioner’s guide to efficient AI

Vivek KalyanaranganManning, early access (MEAP)

Efficient Processing of Deep Neural Networks

Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, Joel S. Emer2020

Vector Quantization and Signal Compression

Allen Gersho, Robert M. Gray1992

Machine Learning Systems

Vijay Janapa Reddi and contributorsOpen-access online textbook

In 2 lists

Related Repositories >Quantization and training toolkits

TorchAO

PyTorch-native quantization for training and inference.

In 2 lists

bitsandbytes

Low-bit linear layers and quantized optimizers, including implementations used by LLM.int8() and QLoRA.

In 4 listsDetails

LLM Compressor

Model compression and quantization workflows for deployment with vLLM.

In 2 lists

NVIDIA Model Optimizer

Quantization and model optimization with export to supported inference runtimes.

In 3 lists

LightCompress (formerly LLMC)

Research and deployment toolkit spanning LLMs, vision-language and generative models.

HQQ

Half-quadratic weight quantization without calibration data.

AIMET

Post-training and quantization-aware model optimization.

Brevitas

PyTorch quantization-aware training with configurable quantizers and hardware export.

Related Repositories >Inference and hardware

llama.cpp

Local LLM inference with GGUF models and multiple quantization formats.

In 12 listsDetails

vLLM

LLM serving with supported low-bit kernels and quantized KV caches.

In 11 listsDetails

TensorRT LLM

NVIDIA GPU inference with supported low-precision formats and optimized kernels.

In 8 listsDetails

Transformer Engine

Low-precision transformer computation, including FP8 and FP4 on supported NVIDIA GPUs.

In 2 lists

Nunchaku

Low-bit diffusion inference, including SVDQuant kernels.

BitNet

Inference framework for supported native low-bit BitNet models.

In 5 listsDetails

FINN

Dataflow compilation for quantized neural networks on FPGAs.

In 2 lists

ncnn

Mobile neural network inference, including INT8 deployment.

In 6 listsDetails
See category
99

Awesome DeepSeek Harness (DSH) Plugin

awesome-dsh-plugin/awesome-dsh-plugin

A curated list of plugins for DeepSeek Harness (dsh) · DeepSeek Harness 插件精选列表

Fresh★ 25k1416 entriesPushed today
98

Awesome OpenClaw Skills

VoltAgent/awesome-openclaw-skills

The awesome collection of OpenClaw skills. 5,400+ skills filtered and categorized from the official OpenClaw Skills Registry.🦞

Fresh★ 54k843 entriesPushed 16 days ago
96

Awesome Prompts

ai-boost/awesome-prompts

Curated prompts, frameworks and papers for LLMs and agents, with an engineering bias. Prompt engineering, context engineering, agent harnesses, red teaming.

Fresh★ 9.3k365 entriesPushed today
96

Awesome Open Source Alternatives

piotrkulpinski/open-source-alternatives

Curated list of open source alternatives to proprietary software.

Fresh★ 6.9k860 entriesPushed today
95

Awesome README

matiassingers/awesome-readme

A curated list of awesome READMEs

Fresh★ 22k144 entriesPushed 5 days ago
94

Awesome Persona Distill Skills

xixu-me/awesome-persona-distill-skills

Curated list of Agent Skills centered on people, relationships, commemorative scenes, and methodological perspectives

Fresh★ 4.7k72 entriesPushed today