Skip to content
41

Awesome XAI

Awesome Explainable AI (XAI) and Interpretable ML Papers and Resources

200 stars16 forks101 entriesLast push May 4, 2021 (5 years ago)License Other

This page lists names, links and short descriptions. The original list on GitHub is the source and belongs to its authors.

Papers >Landmarks

Explanation in Artificial Intelligence: Insights from the Social Sciences

This paper provides an introduction to the social science research into explanations. The author provides 4 major findings: (1) explanations are constrastive, (2) explanations are selected, (3) probabilities probably don't matter, (4) explanations are social. These fit into the general theme that…

Sanity Checks for Saliency Maps

An important read for anyone using saliency maps. This paper proposes two experiments to determine whether saliency maps are useful: (1) model parameter randomization test compares maps from trained and untrained models, (2) data randomization test compares maps from models trained on the original…

Papers >Surveys

Explainable Deep Learning: A Field Guide for the Uninitiated

An in-depth description of XAI focused on technqiues for deep learning.

Papers >Evaluations

Quantifying Explainability of Saliency Methods in Deep Neural Networks

An analysis of how different heatmap-based saliency methods perform based on experimentation with a generated dataset.

Papers >XAI Methods

Ada-SISE

Adaptive semantice inpute sampling for explanation.

ALE

Accumulated local effects plot.

Link may be dead

ALIME

Autoencoder Based Approach for Local Interpretability.

Anchors

High-Precision Model-Agnostic Explanations.

Auditing

Auditing black-box models.

BayLIME

Bayesian local interpretable model-agnostic explanations.

Break Down

Break down plots for additive attributions.

CAM

Class activation mapping.

CDT

Confident interpretation of Bayesian decision tree ensembles.

CICE

Centered ICE plot.

CMM

Combined multiple models metalearner.

Link may be dead

Conj Rules

Using sampling and queries to extract rules from trained neural networks.

Link may be dead

CP

Contribution propogation.

DecText

Extracting decision trees from trained neural networks.

Link may be dead

DeepLIFT

Deep label-specific feature learning for image annotation.

DTD

Deep Taylor decomposition.

Link may be dead

ExplainD

Explanations of evidence in additive classifiers.

Link may be dead

FIRM

Feature importance ranking measure.

Fong, et. al.

Meaninful perturbations model.

G-REX

Rule extraction using genetic algorithms.

Link may be dead

Gibbons, et. al.

Explain random forest using decision tree.

GoldenEye

Exploring classifiers by randomization.

GPD

Gaussian process decisions.

GPDT

Genetic program to evolve decision trees.

GradCAM

Gradient-weighted Class Activation Mapping.

GradCAM++

Generalized gradient-based visual explanations.

Hara, et. al.

Making tree ensembles interpretable.

ICE

Individual conditional expectation plots.

Link may be dead

IG

Integrated gradients.

inTrees

Interpreting tree ensembles with inTrees.

IOFP

Iterative orthoganol feature projection.

In 2 lists

IP

Information plane visualization.

KL-LIME

Kullback-Leibler Projections based LIME.

Krishnan, et. al.

Extracting decision trees from trained neural networks.

Link may be dead

Lei, et. al.

Rationalizing neural predictions with generator and encoder.

LIME

Local Interpretable Model-Agnostic Explanations.

Link may be dead

LOCO

Leave-one covariate out.

Link may be dead

LORE

Local rule-based explanations.

Lou, et. al.

Accurate intelligibile models with pairwise interactions.

Link may be dead

LRP

Layer-wise relevance propogation.

MCR

Model class reliance.

MES

Model explanation system.

MFI

Feature importance measure for non-linear algorithms.

NID

Neural interpretation diagram.

Link may be dead

OptiLIME

Optimized LIME.

PALM

Partition aware local model.

Link may be dead

PDA

Prediction Difference Analysis: Visualize deep neural network decisions.

PDP

Partial dependence plots.

POIMs

Positional oligomer importance matrices for understanding SVM signal detectors.

Link may be dead

ProfWeight

Transfer information from deep network to simpler model.

Prospector

Interactive partial dependence diagnostics.

Link may be dead

QII

Quantitative input influence.

REFNE

Extracting symbolic rules from trained neural network ensembles.

Link may be dead

RETAIN

Reverse time attention model.

RISE

Randomized input sampling for explanation.

RxREN

Reverse engineering neural networks for rule extraction.

SHAP

A unified approach to interpretting model predictions.

SIDU

Similarity, difference, and uniqueness input perturbation.

Simonynan, et. al

Visualizing CNN classes.

Singh, et. al

Programs as black-box explanations.

In 2 lists

STA

Interpreting models via Single Tree Approximation.

Strumbelj, et. al.

Explanation of individual classifications using game theory.

SVM+P

Rule extraction from support vector machines.

Link may be dead

TCAV

Testing with concept activation vectors.

Tolomei, et. al.

Interpretable predictions of tree-ensembles via actionable feature tweaking.

Link may be dead

Tree Metrics

Making sense of a forest of trees.

Link may be dead

TreeSHAP

Consistent feature attribute for tree ensembles.

TreeView

Feature-space partitioning.

TREPAN

Extracting tree-structured representations of trained networks.

TSP

Tree space prototypes.

Link may be dead

VBP

Visual back-propagation.

VEC

Variable effect characteristic curve.

VIN

Variable interaction network.

Link may be dead

X-TREPAN

Adapted etraction of comprehensible decision tree in ANNs.

Xu, et. al.

Show, attend, tell attention model.

Papers >Interpretable Models

Decision List

Like a decision tree with no branches.

Decision Trees

The tree provides an interpretation.

In 3 lists

Explainable Boosting Machine

Method that predicts based on learned vector graphs of features.

k-Nearest Neighbors

The prototypical clustering method.

In 2 lists

Linear Regression

Easily plottable and understandable regression.

In 3 lists

Logistic Regression

Easily plottable and understandable classification.

In 3 lists

Naive Bayes

Good classification, poor estimation using conditional probabilities.

In 4 listsDetails

RuleFit

Sparse linear model as decision rules including feature interactions.

Papers >Critiques

Attention is not Explanation

Authors perform a series of NLP experiments which argue attention does not provide meaningful explanations. They also demosntrate that different attentions can generate similar model outputs.

Attention is not --not-- Explanation

This is a rebutal to the above paper. Authors argue that multiple explanations can be valid and that the and that attention can produce a valid explanation, if not -the- valid explanation.

Do Not Trust Additive Explanations

Authors argue that addditive explanations (e.g. LIME, SHAP, Break Down) fail to take feature ineractions into account and are thus unreliable.

Please Stop Permuting Features An Explanation and Alternatives

Authors demonstrate why permuting features is misleading, especially where there is strong feature dependence. They offer several previously described alternatives.

Stop Explaining Black Box Machine Learning Models for High States Decisions and Use Interpretable Models Instead

Authors present a number of issues with explainable ML and challenges to interpretable ML: (1) constructing optimal logical models, (2) constructing optimal sparse scoring systems, (3) defining interpretability and creating methods for specific methods. They also offer an argument for why…

The (Un)reliability of Saliency Methods

Authors demonstrate how saliency methods vary attribution when adding a constant shift to the input data. They argue that methods should fulfill input invariance, that a saliency method mirror the sensistivity of the model with respect to transformations of the input.

Repositories

EthicalML/xai

A toolkit for XAI which is focused exclusively on tabular data. It implements a variety of data and model evaluation techniques.

MAIF/shapash

SHAP and LIME-based front-end explainer.

In 4 listsDetails

PAIR-code/what-if-tool

A tool for Tensorboard or Notebooks which allows investigating model performance and fairness.

In 2 lists

slundberg/shap

A Python module for using Shapley Additive Explanations.

In 3 lists

Videos

Debate: Interpretability is necessary for ML

A debate on whether interpretability is necessary for ML with Rich Caruana and Patrice Simard for and Kilian Weinberger and Yann LeCun against.

Follow

The Institute for Ethical AI & Machine Learning

A UK-based research center that performs research into ethical AI/ML, which frequently involves XAI.

Tim Miller

One of the preeminent researchers in XAI.

Link may be dead

Rich Caruana

The man behind Explainable Boosting Machines.

See category
100

Table of Contents

hesreallyhim/awesome-claude-code

A hand-picked collection of the finest of resources for the most awesome of agents, Claude Code, the undisputed champion of coding companions, from the unstoppable team…

Fresh★ 58k252 entriesPushed today
100

Awesome Agent Skills

VoltAgent/awesome-agent-skills

A curated collection of 1000+ agent skills from official dev teams and the community, compatible with Claude Code, Codex, Gemini CLI, Cursor, and more.

Fresh★ 37k965 entriesPushed today
98

Awesome Production Machine Learning

EthicalML/awesome-production-machine-learning

A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning

Fresh★ 21k535 entriesPushed 4 days ago
96

Static Analysis

analysis-tools-dev/static-analysis

⚙️ A curated list of static analysis (SAST) tools and linters for all programming languages, config files, build tools, and more. The focus is on tools which improve…

Fresh★ 15k532 entriesPushed 3 days ago
96

Awesome Nano Banana Pro Prompts

YouMind-OpenLab/awesome-nano-banana-pro-prompts

🍌 World's largest Nano Banana Pro prompt library — 10,000+ curated prompts with preview images, 16 languages. Google Gemini AI image generation. Free & open source.

Fresh★ 14k173 entriesPushed today
95

Contents

ossu/computer-science

🎓 Path to a free self-taught education in Computer Science!

Fresh★ 211k65 entriesPushed 7 days ago