Skip to content
87

Awesome Speaker Diarization

A curated list of awesome Speaker Diarization papers, libraries, datasets, and other resources.

1.9k stars254 forks196 entriesLast push Oct 26, 2026 (20 days ago)License Apache-2.0

This page lists names, links and short descriptions. The original list on GitHub is the source and belongs to its authors.

Software >Framework

FunASR

Industrial-grade speech recognition toolkit with built-in speaker diarization (cam++), VAD, ASR (SenseVoice/Paraformer), and punctuation. 50+ languages, 170x realtime.

In 12 listsDetails

NVIDIA NeMo Speech

Scalable generative AI and speech toolkit featuring end-to-end and modular speaker diarization (Sortformer, MSDD, TitaNet speaker embeddings, MarbleNet VAD, and joint ASR-diarization pipelines).

In 4 listsDetails

WhisperX

Fast automatic speech recognition with word-level timestamps (phoneme forced alignment) and speaker diarization via pyannote-audio.

In 4 lists

ESPnet

End-to-end speech processing toolkit including neural speaker diarization (EEND, EEND-EDA) and joint speech enhancement/separation and diarization recipes.

In 4 listsDetails

3D-Speaker

Open-source toolkit for single- and multi-modal (audio-visual) speaker verification, recognition, and speaker diarization (CAM++, ERes2Net, overlap detection, and ONNX runtime).

MiniVox

MiniVox is an open-source evaluation system for the online speaker diarization task.

SpeechBrain

SpeechBrain is an open-source and all-in-one speech toolkit based on PyTorch.

In 5 listsDetails

SIDEKIT for diarization (s4d)

An open-source package extension of SIDEKIT for speaker diarization (source).

pyAudioAnalysis

Python Audio Analysis Library: Feature Extraction, Classification, Segmentation and Applications.

In 4 lists

AaltoASR

Speaker diarization scripts, based on AaltoASR.

LIUM SpkDiarization

LIUM_SpkDiarization is software dedicated to speaker diarization (i.e. speaker segmentation and clustering). It is written in Java, and includes developments in the domain (as of 2013).

kaldi-asr

Example scripts for speaker diarization on a portion of CALLHOME used in the 2000 NIST speaker recognition evaluation.

In 5 listsDetails

kaldi-speaker-diarization

Icelandic speaker diarization scripts using kaldi.

Alize LIA_SpkSeg

ALIZÉ is an open-source platform for speaker recognition; LIA_SpkSeg provides the tools for speaker diarization.

pyannote-audio

Neural building blocks for speaker diarization: speech activity detection, speaker change detection, speaker embedding.

In 3 lists

pyBK

Speaker diarization using binary key speaker modelling. Computationally light solution that does not require external training data.

Speaker-Diarization

Speaker diarization using uis-rnn and GhostVLAD. An easier way to support openset speakers.

EEND

End-to-End Neural Diarization.

VBx

Variational Bayes HMM over x-vectors diarization. x-vector extractor recipe

RE-VERB

RE: VERB is speaker diarization system, it allows the user to send/record audio of a conversation and receive timestamps of who spoke when.

diart

Real-time/streaming speaker diarization extending pyannote.audio to online processing (formerly StreamingSpeakerDiarization).

simple_diarizer

Simplified diarization pipeline using pretrained models. Made to be as simple as possible to go from an input audio file to diarized segments.

Picovoice Falcon

A lightweight, accurate, and fast speaker diarization engine written in C and available in Python, running on CPU with minimal overhead.

DiaPer

PyTorch implementation for DiaPer: End-to-End Neural Diarization with Perceiver-Based Attractors including models pre-trained on free and public data.

sherpa-onnx

Supports speaker diarization, speech recognition, and text-to-speech on various platforms with various language bindings.

In 4 lists

FluidAudio

A native Swift speaker diarization library for Apple platforms, using CoreML for efficient, real-time audio processing with high accuracy.

In 4 listsDetails

speakrs

Rust implementation of the pyannote community-1 diarization pipeline (segmentation, embedding, PLDA, VBx clustering) with native CPU, CUDA, and CoreML backends. 7.0% DER on VoxConverse dev at 1912x realtime on an RTX 4090.

diarizers

Unified pipeline for ASR, standalone speaker diarization (neural segmentation, ECAPA-TDNN speaker embeddings, spectral clustering), and DiarizationLM post-processing, with an interactive Hugging Face Space.

Software >Evaluation

pyannote-metrics

A toolkit for reproducible evaluation, diagnostic, and error analysis of speaker diarization systems.

SimpleDER

A lightweight library to compute Diarization Error Rate (DER), Jaccard Error Rate (JER), and Word Diarization Error Rate (WDER).

DiarizationLM

Implements Word Error Rate (WER), Word Diarization Error Rate (WDER), and concatenated minimum-permutation Word Error Rate (cpWER).

dscore

Diarization scoring tools.

Sequence Match Accuracy

Match the accuracy of two sequences with Hungarian algorithm.

In 2 lists

spyder

Simple Python package for fast DER computation.

CDER

Conversational DER from The Conversational Short-phrase Speaker Diarization (CSSD) Task: Dataset, Evaluation Metric and Baselines

Selective Collar

Selective collar for mitigating evaluation bias in speaker diarization from Beyond Uniform Forgiveness: Introducing the Selective Collar to Mitigate Evaluation Bias in Speaker Diarization

Software >Clustering

Sequence Match Accuracy

Match the accuracy of two sequences with Hungarian algorithm.

In 2 lists

uis-rnn-sml

A variant of UIS-RNN, for the paper Supervised Online Diarization with Sample Mean Loss for Multi-Domain Data.

DNC

Transformer-based Discriminative Neural Clustering (DNC) for Speaker Diarisation. Like UIS-RNN, it is supervised.

SpectralCluster

Spectral clustering with affinity matrix refinement operations, auto-tune, and speaker turn constraints.

sklearn.cluster

scikit-learn clustering algorithms.

In 2 lists

PLDA

Probabilistic Linear Discriminant Analysis & classification, written in Python.

PLDA

Open-source implementation of simplified PLDA (Probabilistic Linear Discriminant Analysis).

Auto-Tuning Spectral Clustering

Auto-tuning Spectral Clustering method that does not need development set or supervised tuning.

Software >Speaker embedding

resemble-ai/Resemblyzer

PyTorch implementation of generalized end-to-end loss for speaker verification, which can be used for voice cloning and diarization.

Speaker_Verification

TensorFlow implementation of generalized end-to-end loss for speaker verification.

PyTorch_Speaker_Verification

PyTorch implementation of "Generalized End-to-End Loss for Speaker Verification" by Wan, Li et al. With UIS-RNN integration.

Real-Time Voice Cloning

Implementation of "Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis" (SV2TTS) with a vocoder that works in real-time.

In 2 lists

conformer-speaker-encoder

Massively multilingual conformer-based speaker recognition models in TFLite format.

deep-speaker

Third party implementation of the Baidu paper Deep Speaker: an End-to-End Neural Speaker Embedding System.

x-vector-kaldi-tf

TensorFlow implementation of x-vector topology on top of Kaldi recipe.

kaldi-ivector

Extension to Kaldi implementing the standard i-vector hyperparameter estimation and i-vector extraction procedure.

voxceleb-ivector

Voxceleb1 i-vector based speaker recognition system.

pytorch_xvectors

PyTorch implementation of Voxceleb x-vectors. Additionally, includes meta-learning architectures for embedding training. Evaluated with speaker diarization and speaker verification.

ASVtorch

ASVtorch is a toolkit for automatic speaker recognition.

asv-subtools

ASV-Subtools is developed based on PyTorch and Kaldi for the task of speaker recognition, language identification, etc. The 'sub' of 'subtools' means that there are many modular tools and the parts constitute the whole.

WeSpeaker

WeSpeaker is a research and production oriented speaker verification, recognition and diarization toolkit, which supports very strong recipes with on-the-fly data preparation, model training and evaluation, as well as runtime C++ codes.

ReDimNet

Neural network architecture presented in the paper Reshape Dimensions Network for Speaker Recognition

Software >Speaker change detection

change_detection

Code for Speaker Change Detection in Broadcast TV using Bidirectional Long Short-Term Memory Networks.

tinydiarize

Diarization inside OpenAI Whisper decoder

Software >Audio feature extraction

LibROSA

Python library for audio and music analysis. https://librosa.github.io/

In 5 listsDetails

python_speech_features

This library provides common speech features for ASR including MFCCs and filterbank energies. https://python-speech-features.readthedocs.io/en/latest/

In 3 lists

pyAudioAnalysis

Python Audio Analysis Library: Feature Extraction, Classification, Segmentation and Applications.

In 4 lists

Software >Audio data augmentation

pyroomacoustics

Pyroomacoustics is a package for audio signal processing for indoor applications. It was developed as a fast prototyping platform for beamforming algorithms in indoor scenarios. https://pyroomacoustics.readthedocs.io

In 2 lists

gpuRIR

Python library for Room Impulse Response (RIR) simulation with GPU acceleration

rir_simulator_python

Room impulse response simulator using python

WavAugment

WavAugment performs data augmentation on audio data. The audio data is represented as PyTorch tensors.

EEND_dataprep

Recipes for generating simulated conversations used to train end-to-end diarization models.

Software >Other software

VB Diarization

VB Diarization with Eigenvoice and HMM Priors.

DOVER-Lap

Python package for combining diarization system outputs

Diar-az

Data formatting tool to support the ruv-di dataset. Kaldi to Gecko to Kaldi and corpus and back.

voxsolo

Verbatim per-speaker audio from diarization: keeps one speaker bit-exact, silences other speakers and overlapped speech; exports EDL/SRT/VTT/Audacity labels for editors.

VoxLock

Streaming target speaker extraction: keeps one speaker's voice and removes the other voices in the room before speech recognition. 27M parameters, 24 ms algorithmic latency on CPU, ONNX and safetensors weights.

OATS

Open-source macOS app for meeting transcription and notes with speaker diarization (FluidAudio) and optional fully offline mode.

In 4 listsDetails

akou

Open-source macOS app that records calls and meetings and transcribes them on-device, with speaker labels from NVIDIA Nemotron 3 Diarization.

In 3 lists

Local Transcriber

macOS menu bar app that records calls and meetings and transcribes them on-device (Whisper large-v3-turbo on MLX), with pyannote diarization per audio track and a voice database that recognizes known speakers.

In 3 lists

Datasets >Diarization datasets

2000 NIST Speaker Recognition Evaluation

Evaluation Plan

2003 NIST Rich Transcription Evaluation Data

Telephone speech, broadcast news.

CALLHOME American English Speech

CH109 whitelist

The ICSI Meeting Corpus

~72 hours (75 meetings). License (CC BY 4.0)

The AMI Meeting Corpus

100 hours of multi-accent English meetings. License (CC BY 4.0)

Fisher English Training Speech Part 1 Speech

Conversational telephone speech (5,850 calls, ~975 hours).

Fisher English Training Part 2, Speech

Conversational telephone speech (5,849 calls, ~975 hours).

DIHARD III

68.15 hours (34.15h dev + 34.00h eval) across 11 diverse acoustic domains (clinical, courtroom, restaurant, web video, CTS, etc.).

VoxConverse

Audio-visual diarisation dataset of 63.8 hours across 448 multispeaker YouTube clips (CC BY 4.0).

CHiME-6

~50 hours (20 4-speaker dinner-party sessions) recorded on distant Kinect microphone arrays and binaural microphones (OpenSLR 150).

AISHELL-4

120 hours (211 sessions, 4–8 speakers per session) of real Mandarin multi-party meetings recorded on an 8-channel circular mic array + headsets (CC BY-SA 4.0).

MiniVox

MiniVox is an open-source evaluation system for the online speaker diarization task.

The AliMeeting Corpus

118.75 hours of real Mandarin meetings (far-field 8-ch mic array + near-field headsets, ~42% overlap in train; CC BY-SA 4.0). Baseline repo.

Datasets >Speaker embedding training sets

TIMIT

Published in 1993, the TIMIT corpus of read speech is one of the earliest speaker recognition datasets.

VCTK

CSTR VCTK Corpus (v0.92, CC BY 4.0). Newspaper sentences plus the Rainbow Passage and an elicitation paragraph across diverse English accents.

LibriSpeech

Large-scale (~1,000 hours) corpus of read English speech (CC BY 4.0).

Multilingual LibriSpeech (MLS)

Large-scale (~50.8K hours) multilingual corpus derived from read audiobooks from LibriVox across 8 languages - English, German, Dutch, Spanish, French, Italian, Portuguese, Polish (CC BY 4.0).

LibriVox

Free public-domain audiobooks (21.8K+ cataloged works across 49 languages). LibriSpeech and MLS are processed subsets of LibriVox. Each unsegmented chapter recording can be very long.

In 2 lists

VoxCeleb 1&2

Audio-visual dataset (~2,794 hours) consisting of short clips of human speech extracted from YouTube interview videos (CC BY-SA 4.0).

The Spoken Wikipedia Corpora

Volunteer readers reading full Wikipedia articles (~1,005 hours total, CC BY-SA 4.0).

CN-Celeb 1&2

Multi-genre Chinese speaker recognition corpus (~1,364 hours across 11 genres: CN-Celeb1 + CN-Celeb2; CC BY-SA 4.0) released by CSLT@Tsinghua University.

BookTubeSpeech

Audio samples (~1,090 hours) extracted from BookTube YouTube videos (CC BY 4.0). The dataset can be downloaded using BookTubeSpeech-download.

DeepMine

A speech database in Persian and English (>480 hours transcribed) designed to build and evaluate speaker verification and Persian ASR systems.

NISP-Dataset

Multi-accent Indian English and 5 Indian languages (56.9 hours, CC BY 4.0) along with speaker physical parameters (height, weight, ...) and regional/linguistic metadata.

VoxBlink2

Multilingual dataset (~16,672 hours, CC BY-NC-SA 4.0) from VoxBlink2: A 100K+ Speaker Recognition Corpus and the Open-Set Speaker-Identification Benchmark

Datasets >Augmentation noise sources

AudioSet

A large-scale dataset of 2.08M human-annotated 10-second audio event clips (~5,789 hours; CC BY 4.0).

MUSAN

Corpus of music (42.5h), speech (~60h), and noise (~6h, 929 files) recordings (CC BY 4.0).

RIRS_NOISES

Simulated and real room impulse responses plus isotropic and point-source noises at 16 kHz (Apache 2.0).

Publications >Special topics

A Review of Speaker Diarization: Recent Advances with Deep Learning

, 2022

A review on speaker diarization systems and approaches

, 2012

Link may be dead

Speaker diarization: A review of recent research

, 2012

SpeakerLM: End-to-End Versatile Speaker Diarization and Recognition with Multimodal Large Language Models

, 2026

DiarizationLM: Speaker Diarization Post-Processing with Large Language Models

, 2024

Enhancing Speaker Diarization with Large Language Models: A Contextual Beam Search Approach

, 2024

Lexical speaker error correction: Leveraging language models for speaker diarization error correction

, 2023

DiaPer: End-to-End Neural Diarization with Perceiver-Based Attractors

, 2024

Powerset multi-class cross entropy loss for neural speaker diarization

, 2023

TOLD: A Novel Two-Stage Overlap-Aware Framework for Speaker Diarization

, 2023

Encoder-Decoder Based Attractors for End-to-End Neural Diarization

, 2022

Speaker Overlap-aware Neural Diarization for Multi-party Meeting Analysis

, 2022

End-to-End Diarization for Variable Number of Speakers with Local-Global Networks and Discriminative Speaker Embeddings

, 2021

Discriminative Neural Clustering for Speaker Diarisation

, 2021

Supervised online diarization with sample mean loss for multi-domain data

, 2020

End-to-End Neural Speaker Diarization with Permutation-Free Objectives

, 2019

End-to-End Neural Speaker Diarization with Self-attention

, 2019

Fully Supervised Speaker Diarization

, 2019

Train Short, Infer Long: Speech-LLM Enables Zero-Shot Streamable Joint ASR and Diarization on Long Audio

, 2026

Sortformer: A Novel Approach for Permutation-Resolved Speaker Supervision in Speech-to-Text Systems

, 2025

One Model to Rule Them All? Towards End-to-End Joint Speaker Diarization and Speech Recognition

, 2024

WhisperX: Time-Accurate Speech Transcription of Long-Form Audio

, 2023

A Comparative Study on Speaker-attributed Automatic Speech Recognition in Multi-party Meetings

, 2022

Turn-to-Diarize: Online Speaker Diarization Constrained by Transformer Transducer Speaker Turn Detection

, 2022

Transcribe-to-Diarize: Neural Speaker Diarization for Unlimited Number of Speakers using End-to-End Speaker-Attributed…

, 2022

Joint Speech Recognition and Speaker Diarization via Sequence Transduction

, 2019

Says who? Deep learning models for joint speech recognition, segmentation and diarization

, 2018

Speaker Diarization as a Fully Online Bandit Learning Problem in MiniVox

, 2021

VoiceID on the Fly: A Speaker Recognition System that Learns from Scratch

, 2020

The CHiME-8 DASR Challenge for Generalizable and Array Agnostic Distant Automatic Speech Recognition and Diarization

, 2024

M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge

, 2022

The Hitachi-JHU DIHARD III system: Competitive end-to-end neural diarization and x-vector clustering systems combined…

, 2021

Diarization is Hard: Some Experiences and Lessons Learned for the JHU Team in the Inaugural DIHARD Challenge

, 2018

ODESSA at Albayzin Speaker Diarization Challenge 2018

, 2018

Joint Discriminative Embedding Learning, Speech Activity and Overlap Detection for the DIHARD Challenge

, 2018

3D-Speaker-Toolkit: An Open-Source Toolkit for Multimodal Speaker Verification and Diarization

, 2025

AVA-AVD: Audio-Visual Speaker Diarization in the Wild

, 2022

Link may be dead

DyViSE: Dynamic Vision-Guided Speaker Embedding for Audio-Visual Speaker Diarization

, 2022

End-to-End Audio-Visual Neural Speaker Diarization

, 2022

MSDWild: Multi-modal Speaker Diarization Dataset in the Wild

, 2022

Publications >Other

Efficient and Robust Speaker Diarization via Structured Pruning of Self-Supervised Models

Leveraging Self-Supervised Learning for Speaker Diarization

TS-SEP: Joint Diarization and Separation Conditioned on Estimated Speaker Embeddings

pyannote.audio 2.1 speaker diarization pipeline: principle, benchmark, and recipe

From Simulated Mixtures to Simulated Conversations as Training Data for End-to-End Neural Diarization

Multi-scale Speaker Diarization with Dynamic Scale Weighting

Overlap-aware low-latency online speaker diarization based on end-to-end local segmentation

End-to-end speaker segmentation for overlap-aware resegmentation

DIVE: End-to-end Speech Diarization via Iterative Speaker Embedding

DOVER-Lap: A method for combining overlap-aware diarization outputs

Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: Theory, implementation and analysis on…

Link may be dead

AISHELL-4: An Open Source Dataset for Speech Enhancement, Separation, Recognition and Speaker Diarization in…

Spot the conversation: speaker diarisation in the wild

Speaker Diarization with Region Proposal Network

Target-Speaker Voice Activity Detection: a Novel Approach for Multi-Speaker Diarization in a Dinner Party Scenario

Overlap-aware diarization: resegmentation using neural end-to-end overlapped speech detection

Speaker diarization using latent space clustering in generative adversarial network

A study of semi-supervised speaker diarization system using gan mixture model

Enhancements for Audio-only Diarization Systems

LSTM based Similarity Measurement with Spectral Clustering for Speaker Diarization

Meeting Transcription Using Virtual Microphone Arrays

Speaker diarisation using 2D self-attentive combination of embeddings

Speaker Diarization with Lexical Information

Speaker Diarization with LSTM

Neural speech turn segmentation and affinity propagation for speaker diarization

Multimodal Speaker Segmentation and Diarization using Lexical and Acoustic Cues via Sequence to Sequence Neural Networks

Speaker diarization using deep neural network embeddings

Speaker diarization using convolutional neural network for statistics accumulation refinement

pyannote.metrics: a toolkit for reproducible evaluation, diagnostic, and error analysis of speaker diarization systems

Speaker Change Detection in Broadcast TV using Bidirectional Long Short-Term Memory Networks

Speaker Diarization using Deep Recurrent Convolutional Neural Networks for Speaker Embeddings

A Speaker Diarization System for Studying Peer-Led Team Learning Groups

Diarization resegmentation in the factor analysis subspace

A study of the cosine distance-based mean shift for telephone speech diarization

Speaker diarization with PLDA i-vector scoring and unsupervised calibration

Artificial neural network features for speaker diarization

Unsupervised methods for speaker diarization: An integrated and iterative approach

PLDA-based Clustering for Speaker Diarization of Broadcast Streams

Speaker diarization of meetings based on speaker role n-gram models

Speaker Diarization for Meeting Room Audio

Stream-based speaker segmentation using speaker factors and eigenvoices

An overview of automatic speaker diarization systems

A spectral clustering approach to speaker diarization

Other learning materials >Books

Voice Identity Techniques: From core algorithms to engineering practice (Chinese)

by Quan Wang, 2020

Other learning materials >Tech blogs

Literature Review For Speaker Change Detection

by Halil Erdoğan

Speaker Diarization: Separation of Multiple Speakers in an Audio File

by Jaspreet Singh

Link may be dead

Speaker Diarization with Kaldi

by Yoav Ramon

Link may be dead

Who spoke when! How to Build your own Speaker Diarization Module

by Rahul Saxena

Link may be dead

Other learning materials >Video tutorials

pyannote audio: neural building blocks for speaker diarization

by Hervé Bredin

Google's Diarization System: Speaker Diarization with LSTM

by Google

Fully Supervised Speaker Diarization: Say Goodbye to clustering

by Google

Turn-to-Diarize: Online Speaker Diarization Constrained by Transformer Transducer Speaker Turn Detection

by Google

Speaker Diarization: Optimal Clustering and Learning Speaker Embeddings

by Microsoft Research

Robust Speaker Diarization for Meetings: the ICSI system

by Microsoft Research

【机器之心&博文视点】入门声纹技术|第二讲:声纹分割聚类与其他应用

by Quan Wang

See category
100

Table of Contents

hesreallyhim/awesome-claude-code

A hand-picked collection of the finest of resources for the most awesome of agents, Claude Code, the undisputed champion of coding companions, from the unstoppable team…

Fresh★ 58k252 entriesPushed today
100

Awesome Agent Skills

VoltAgent/awesome-agent-skills

A curated collection of 1000+ agent skills from official dev teams and the community, compatible with Claude Code, Codex, Gemini CLI, Cursor, and more.

Fresh★ 37k965 entriesPushed today
98

Awesome Production Machine Learning

EthicalML/awesome-production-machine-learning

A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning

Fresh★ 21k535 entriesPushed 4 days ago
96

Static Analysis

analysis-tools-dev/static-analysis

⚙️ A curated list of static analysis (SAST) tools and linters for all programming languages, config files, build tools, and more. The focus is on tools which improve…

Fresh★ 15k532 entriesPushed 3 days ago
96

Awesome Nano Banana Pro Prompts

YouMind-OpenLab/awesome-nano-banana-pro-prompts

🍌 World's largest Nano Banana Pro prompt library — 10,000+ curated prompts with preview images, 16 languages. Google Gemini AI image generation. Free & open source.

Fresh★ 14k173 entriesPushed today
95

Contents

ossu/computer-science

🎓 Path to a free self-taught education in Computer Science!

Fresh★ 211k65 entriesPushed 7 days ago