Interpretability, Explainable AI

Interpretable Machine Learning. Overview of different methods:

• Interpretable Machine Learning Christoph Molnar
• Explanatory Model Analysis Przemyslaw Biecek, Tomasz Burzykowski

Interpretable models

• RETAIN (Code)
  • RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism (2016) Edward Choi, Mohammad Taha Bahadori, Joshua A. Kulas, Andy Schuetz, Walter F. Stewart, Jimeng Sun
  • Interpreting Deep Glucose Predictive Models forDiabetic People Using RETAIN (2020) Maxime De Bois, Mounim A. El Yacoubi, Mehdi Ammi

Local explanations

• LIME. Local Interpretable Model-Agnostic Explanations (Code, CRAN, PyPI)
  • “Why should I trust you?” Explaining the predictions of any classifier (2016) Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin
  • MeLIME: Meaningful Local Explanation forMachine Learning Models (2020) Tiago Botari, Frederik Hvilshøj, Rafael Izbicki, Andre C. P. L. F. de Carvalho

Global explanations

• Feature importance
  • Model-agnostic Feature Importance and Effectswith Dependent Features – A Conditional Subgroup Approach (2020) Christoph Molnar, Gunnar Konig, Bernd Bischl, Giuseppe Casalicchio
• SHAP. Shapley Additive Exlanations (Code, PyPI)
  • A Unified Approach to Interpreting ModelPredictions (2017) Scott M. Lundberg, Su-In Lee
• DeepLift. Deep Learning Important Features (Code, Video)
  • Learning Important Features Through Propagating Activation Differences (2019) Avanti Shrikumar1, Peyton Greenside1, Anshul Kundaje

Timeseries

• TSViz
  • TSViz: Demystification of Deep Learning Models for Time-Series Analysis (2018) Siddiqui S.A., Mercier D., Munir M., Dengel A., Ahmed S.

Critics

• Stop explaining black box machine learning models for high stakes decisions and use interpretablemodels instead (2019) Cynthia Rudin
• Limitations of Interpretable Machine Learning Methods (2020)
• Please Stop Permuting Features. An Explanation and Alternatives (2019) Giles Hooker, Lucas Mentch
• Attention is not Explanation (2019) Sarthak Jain, Byron C. Wallace
• Pitfalls to Avoid when Interpreting Machine Learning Models (2020) Christoph Molnar, Gunnar Konig, Julia Herbinger, Timo Freiesleben, Susanne Dandl, Christian A. Scholbeck, Giuseppe Casalicchio, Moritz Grosse-Wentrup, Bernd Bisch

Software

• Python
  • Captum - model interpretability and understanding for PyTorch (Homepage, Paper)
  • AIX360 (Homepage)
  • Alibi (Homepage, PyPI)
  • Eli5 (Homepage, PyPI)

Star Issue