Machine learning and statistical models

General

• Linear models
  • y = Xβ + ε
  • Common statistical tests are linear models (or: how to teach stats) Jonas Kristoffer Lindeløv
• GLM
  • Generalized Linear Model (1972) J. A. Nelder, R. W. M. Wedderburn
• GLMM Generalized linear mixed-effect model
  • y = Xβ + Zu + ε
• LASSO
• MARS/Earth Multivariate Adaptive Regression Splines
  • y = ΣcⱼBⱼ(x)
• GAM, GA2M, GAMM Additive Models (Wiki)
  • g(E(y)) = β + Σfⱼ(xⱼ) or g(E(y)) = β + Σfⱼ(xⱼ)+ Σfᵢⱼ(xᵢ,xⱼ)
  • Generalized Additive Models (1986) T. Hastie, R. Tibshirani
  • Intelligible Models for Classification and Regression (2012) Yin Lou, Rich Caruana, Johannes Gehrke
  • Accurate Intelligible Models with Pairwise Interactions (2013) Yin Lou, Rich Caruana, Johannes Gehrke, Giles Hooker
  • Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission (2015) Rich Caruana, Paul Koch, Yin Lou, Marc Sturm, Johannes Gehrke, Noemie Elhadad
  • Using a generalized additive model with autoregressive terms to study the effects of daily temperature on mortality (2012) Lei Yang, Guoyou Qin, Guixiang Song
  • Generalized additive neural networks (1999) William J. E. Potts
  • Neural Additive Models:Interpretable Machine Learning with Neural Nets (2020) Rishabh Agarwal, Nicholas Frosst, Xuezhou Zhang, Rich Caruana, Geoffrey E. Hinton
  • How Interpretable and Trustworthy are GAMs? (2020) Chun-Hao Chang, Sarah Tan, Ben Lengerich, Anna Goldenberg, Rich Caruana
• Simpson Paradox (Wiki)
  • Simpson’s Paradox and the implications for medical trials (2019) Norman Fenton, Martin Neil, Anthony Constantinou
• Quantile Regression (Wiki)
  • Quantile Regression Forests (2006) Nicolai Meinshausen
  • Probabilistic forecasting approaches for extreme NO2 episodes: acomparison of models (2020) Sebastian Perez Vasseur, Jose L. Aznarte

Trees

• CART Classification And Regression Tree
  • rpart (CRAN)
  • party (CRAN)
• BART Bayesian Additive Regression Trees### Uplift models

SVM

Survival models

Timeseries

• Forecasting: Principles and Practice (2018) Rob J Hyndman, George Athanasopoulos
• Kalman Filters (Wiki)
  • A new approach to linear filtering and prediction problems (1960) R. E. Kalman
• Hidden Markov Model (Wiki)
  • A tutorial on Hidden Markov Models and selected applications in speech recognition (1989) Lawrence R. Rabiner
  • The Infinite Hidden Markov Model Matthew J. Beal, Zoubin Ghahramani, Carl Edward Rasmussen
• ARIMA Autoregressive Integrated Moving Average
• ARIMAX (Seasonal)
• ARMA-GARCH
• SSM State Space Models
• VAR Vector autoregressions
• Holt-Winters exponential smoothing
• Loess model Seasonal decomposition
• Dynamic regression models
• CHARME Conditional Heteroscedastic Autoregressive Mixture of Experts
• Dynamic time warping Wiki

Systems modeling

The coming century will be dominated by major social, political turmoil. And it will result primarily because people are doing what they think they should do, but do not realize that what they’re doing are causing these problems. So, I think the hope for this coming century is to develop a sufficiently large percentage of the population that have true insight into the nature of the complex systems within which they live. - Jay Forrester

• Logistic Chaos
  • Logistic Equation
• Predator-Prey Lotka–Volterra equations (Wiki)
  • Using predator-prey models on the Canadian lynx series
• Lorenz attractor (Wiki)
• Bass diffusion model (Wiki)
  • Comments on “A New Product Growth for Model Consumer Durables The Bass Model” (2004) Frank M. Bass
• SIR, SEIR
• DICE Dynamic Integrated Climate-Economy model (Wiki)
  • The “Dice” Model: Background and Structure of a Dynamic Integrated Climate-Economy Model of the Economics of Global Warming (1992) W. Nordhaus

Bayesian networks

Ensembles

Neural nets

Star Issue