FunASR
Industrial-grade speech recognition toolkit with built-in speaker diarization (cam++), VAD, ASR (SenseVoice/Paraformer), and punctuation. 50+ languages, 170x realtime.
A curated list of awesome Speaker Diarization papers, libraries, datasets, and other resources.
This page lists names, links and short descriptions. The original list on GitHub is the source and belongs to its authors.
Industrial-grade speech recognition toolkit with built-in speaker diarization (cam++), VAD, ASR (SenseVoice/Paraformer), and punctuation. 50+ languages, 170x realtime.
Scalable generative AI and speech toolkit featuring end-to-end and modular speaker diarization (Sortformer, MSDD, TitaNet speaker embeddings, MarbleNet VAD, and joint ASR-diarization pipelines).
Fast automatic speech recognition with word-level timestamps (phoneme forced alignment) and speaker diarization via pyannote-audio.
End-to-end speech processing toolkit including neural speaker diarization (EEND, EEND-EDA) and joint speech enhancement/separation and diarization recipes.
Open-source toolkit for single- and multi-modal (audio-visual) speaker verification, recognition, and speaker diarization (CAM++, ERes2Net, overlap detection, and ONNX runtime).
MiniVox is an open-source evaluation system for the online speaker diarization task.
SpeechBrain is an open-source and all-in-one speech toolkit based on PyTorch.
An open-source package extension of SIDEKIT for speaker diarization (source).
Python Audio Analysis Library: Feature Extraction, Classification, Segmentation and Applications.
Speaker diarization scripts, based on AaltoASR.
LIUM_SpkDiarization is software dedicated to speaker diarization (i.e. speaker segmentation and clustering). It is written in Java, and includes developments in the domain (as of 2013).
Example scripts for speaker diarization on a portion of CALLHOME used in the 2000 NIST speaker recognition evaluation.
Icelandic speaker diarization scripts using kaldi.
ALIZÉ is an open-source platform for speaker recognition; LIA_SpkSeg provides the tools for speaker diarization.
Neural building blocks for speaker diarization: speech activity detection, speaker change detection, speaker embedding.
Speaker diarization using binary key speaker modelling. Computationally light solution that does not require external training data.
Speaker diarization using uis-rnn and GhostVLAD. An easier way to support openset speakers.
End-to-End Neural Diarization.
Variational Bayes HMM over x-vectors diarization. x-vector extractor recipe
RE: VERB is speaker diarization system, it allows the user to send/record audio of a conversation and receive timestamps of who spoke when.
Real-time/streaming speaker diarization extending pyannote.audio to online processing (formerly StreamingSpeakerDiarization).
Simplified diarization pipeline using pretrained models. Made to be as simple as possible to go from an input audio file to diarized segments.
A lightweight, accurate, and fast speaker diarization engine written in C and available in Python, running on CPU with minimal overhead.
PyTorch implementation for DiaPer: End-to-End Neural Diarization with Perceiver-Based Attractors including models pre-trained on free and public data.
Supports speaker diarization, speech recognition, and text-to-speech on various platforms with various language bindings.
A native Swift speaker diarization library for Apple platforms, using CoreML for efficient, real-time audio processing with high accuracy.
Rust implementation of the pyannote community-1 diarization pipeline (segmentation, embedding, PLDA, VBx clustering) with native CPU, CUDA, and CoreML backends. 7.0% DER on VoxConverse dev at 1912x realtime on an RTX 4090.
Unified pipeline for ASR, standalone speaker diarization (neural segmentation, ECAPA-TDNN speaker embeddings, spectral clustering), and DiarizationLM post-processing, with an interactive Hugging Face Space.
A toolkit for reproducible evaluation, diagnostic, and error analysis of speaker diarization systems.
A lightweight library to compute Diarization Error Rate (DER), Jaccard Error Rate (JER), and Word Diarization Error Rate (WDER).
Implements Word Error Rate (WER), Word Diarization Error Rate (WDER), and concatenated minimum-permutation Word Error Rate (cpWER).
Diarization scoring tools.
Match the accuracy of two sequences with Hungarian algorithm.
Simple Python package for fast DER computation.
Conversational DER from The Conversational Short-phrase Speaker Diarization (CSSD) Task: Dataset, Evaluation Metric and Baselines
Selective collar for mitigating evaluation bias in speaker diarization from Beyond Uniform Forgiveness: Introducing the Selective Collar to Mitigate Evaluation Bias in Speaker Diarization
Match the accuracy of two sequences with Hungarian algorithm.
A variant of UIS-RNN, for the paper Supervised Online Diarization with Sample Mean Loss for Multi-Domain Data.
Transformer-based Discriminative Neural Clustering (DNC) for Speaker Diarisation. Like UIS-RNN, it is supervised.
Spectral clustering with affinity matrix refinement operations, auto-tune, and speaker turn constraints.
scikit-learn clustering algorithms.
Probabilistic Linear Discriminant Analysis & classification, written in Python.
Open-source implementation of simplified PLDA (Probabilistic Linear Discriminant Analysis).
Auto-tuning Spectral Clustering method that does not need development set or supervised tuning.
PyTorch implementation of generalized end-to-end loss for speaker verification, which can be used for voice cloning and diarization.
TensorFlow implementation of generalized end-to-end loss for speaker verification.
PyTorch implementation of "Generalized End-to-End Loss for Speaker Verification" by Wan, Li et al. With UIS-RNN integration.
Implementation of "Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis" (SV2TTS) with a vocoder that works in real-time.
Massively multilingual conformer-based speaker recognition models in TFLite format.
Third party implementation of the Baidu paper Deep Speaker: an End-to-End Neural Speaker Embedding System.
TensorFlow implementation of x-vector topology on top of Kaldi recipe.
Extension to Kaldi implementing the standard i-vector hyperparameter estimation and i-vector extraction procedure.
Voxceleb1 i-vector based speaker recognition system.
PyTorch implementation of Voxceleb x-vectors. Additionally, includes meta-learning architectures for embedding training. Evaluated with speaker diarization and speaker verification.
ASVtorch is a toolkit for automatic speaker recognition.
ASV-Subtools is developed based on PyTorch and Kaldi for the task of speaker recognition, language identification, etc. The 'sub' of 'subtools' means that there are many modular tools and the parts constitute the whole.
WeSpeaker is a research and production oriented speaker verification, recognition and diarization toolkit, which supports very strong recipes with on-the-fly data preparation, model training and evaluation, as well as runtime C++ codes.
Neural network architecture presented in the paper Reshape Dimensions Network for Speaker Recognition
Code for Speaker Change Detection in Broadcast TV using Bidirectional Long Short-Term Memory Networks.
Diarization inside OpenAI Whisper decoder
This library provides common speech features for ASR including MFCCs and filterbank energies. https://python-speech-features.readthedocs.io/en/latest/
Python Audio Analysis Library: Feature Extraction, Classification, Segmentation and Applications.
Pyroomacoustics is a package for audio signal processing for indoor applications. It was developed as a fast prototyping platform for beamforming algorithms in indoor scenarios. https://pyroomacoustics.readthedocs.io
Python library for Room Impulse Response (RIR) simulation with GPU acceleration
Room impulse response simulator using python
WavAugment performs data augmentation on audio data. The audio data is represented as PyTorch tensors.
Recipes for generating simulated conversations used to train end-to-end diarization models.
VB Diarization with Eigenvoice and HMM Priors.
Python package for combining diarization system outputs
Data formatting tool to support the ruv-di dataset. Kaldi to Gecko to Kaldi and corpus and back.
Verbatim per-speaker audio from diarization: keeps one speaker bit-exact, silences other speakers and overlapped speech; exports EDL/SRT/VTT/Audacity labels for editors.
Streaming target speaker extraction: keeps one speaker's voice and removes the other voices in the room before speech recognition. 27M parameters, 24 ms algorithmic latency on CPU, ONNX and safetensors weights.
Open-source macOS app for meeting transcription and notes with speaker diarization (FluidAudio) and optional fully offline mode.
Open-source macOS app that records calls and meetings and transcribes them on-device, with speaker labels from NVIDIA Nemotron 3 Diarization.
macOS menu bar app that records calls and meetings and transcribes them on-device (Whisper large-v3-turbo on MLX), with pyannote diarization per audio track and a voice database that recognizes known speakers.
Evaluation Plan
Telephone speech, broadcast news.
CH109 whitelist
~72 hours (75 meetings). License (CC BY 4.0)
100 hours of multi-accent English meetings. License (CC BY 4.0)
Conversational telephone speech (5,850 calls, ~975 hours).
Conversational telephone speech (5,849 calls, ~975 hours).
68.15 hours (34.15h dev + 34.00h eval) across 11 diverse acoustic domains (clinical, courtroom, restaurant, web video, CTS, etc.).
Audio-visual diarisation dataset of 63.8 hours across 448 multispeaker YouTube clips (CC BY 4.0).
~50 hours (20 4-speaker dinner-party sessions) recorded on distant Kinect microphone arrays and binaural microphones (OpenSLR 150).
120 hours (211 sessions, 4–8 speakers per session) of real Mandarin multi-party meetings recorded on an 8-channel circular mic array + headsets (CC BY-SA 4.0).
MiniVox is an open-source evaluation system for the online speaker diarization task.
118.75 hours of real Mandarin meetings (far-field 8-ch mic array + near-field headsets, ~42% overlap in train; CC BY-SA 4.0). Baseline repo.
Published in 1993, the TIMIT corpus of read speech is one of the earliest speaker recognition datasets.
CSTR VCTK Corpus (v0.92, CC BY 4.0). Newspaper sentences plus the Rainbow Passage and an elicitation paragraph across diverse English accents.
Large-scale (~1,000 hours) corpus of read English speech (CC BY 4.0).
Large-scale (~50.8K hours) multilingual corpus derived from read audiobooks from LibriVox across 8 languages - English, German, Dutch, Spanish, French, Italian, Portuguese, Polish (CC BY 4.0).
Free public-domain audiobooks (21.8K+ cataloged works across 49 languages). LibriSpeech and MLS are processed subsets of LibriVox. Each unsegmented chapter recording can be very long.
Audio-visual dataset (~2,794 hours) consisting of short clips of human speech extracted from YouTube interview videos (CC BY-SA 4.0).
Volunteer readers reading full Wikipedia articles (~1,005 hours total, CC BY-SA 4.0).
Multi-genre Chinese speaker recognition corpus (~1,364 hours across 11 genres: CN-Celeb1 + CN-Celeb2; CC BY-SA 4.0) released by CSLT@Tsinghua University.
Audio samples (~1,090 hours) extracted from BookTube YouTube videos (CC BY 4.0). The dataset can be downloaded using BookTubeSpeech-download.
A speech database in Persian and English (>480 hours transcribed) designed to build and evaluate speaker verification and Persian ASR systems.
Multi-accent Indian English and 5 Indian languages (56.9 hours, CC BY 4.0) along with speaker physical parameters (height, weight, ...) and regional/linguistic metadata.
Multilingual dataset (~16,672 hours, CC BY-NC-SA 4.0) from VoxBlink2: A 100K+ Speaker Recognition Corpus and the Open-Set Speaker-Identification Benchmark
A large-scale dataset of 2.08M human-annotated 10-second audio event clips (~5,789 hours; CC BY 4.0).
Corpus of music (42.5h), speech (~60h), and noise (~6h, 929 files) recordings (CC BY 4.0).
Simulated and real room impulse responses plus isotropic and point-source noises at 16 kHz (Apache 2.0).
by Halil Erdoğan
by Yoav Ramon
by Microsoft Research
by Microsoft Research
by Quan Wang
hesreallyhim/awesome-claude-code
A hand-picked collection of the finest of resources for the most awesome of agents, Claude Code, the undisputed champion of coding companions, from the unstoppable team…
VoltAgent/awesome-agent-skills
A curated collection of 1000+ agent skills from official dev teams and the community, compatible with Claude Code, Codex, Gemini CLI, Cursor, and more.
EthicalML/awesome-production-machine-learning
A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning
analysis-tools-dev/static-analysis
⚙️ A curated list of static analysis (SAST) tools and linters for all programming languages, config files, build tools, and more. The focus is on tools which improve…
YouMind-OpenLab/awesome-nano-banana-pro-prompts
🍌 World's largest Nano Banana Pro prompt library — 10,000+ curated prompts with preview images, 16 languages. Google Gemini AI image generation. Free & open source.
ossu/computer-science
🎓 Path to a free self-taught education in Computer Science!