Additive Models for Regression, Generalized Additive Models, etc.

Last update: 21 Apr 2025 21:17
First version:

Yet Another Inadequate Placeholder, because I don't feel like re-writing a chapter from my textbook. (See below.)

Regression, especially Nonparametric Regression

Andreas Buja, Trevor Hastie and Robert Tibshirani, "Linear smoothers and additive models", Annals of Statistics 17 (1989): 453--510 [A classic additive models paper. The discussions and reply fill pp. 510--555.]
Simon N. Wood, Generalized Additive Models: An Introduction with R

Markus Abel, Karsten Ahnert, Jürgen Kurths and Simon Mandelj, "Additive nonparametric reconstruction of dynamical systems from time series", Physical Review E 71 (2005): 015203
Azadeh Alimadad and Matias Salibian-Barrera, "An Outlier-Robust Fit for Generalized Additive Models with Applications to Disease Outbreak Detection", Journal of the American Statistical Association 106 (2011): 719--731
Tianjiao Chu and Clark Glymour, "Search for Additive Nonlinear Time Series Causal Models", Journal of Machine Learning Research 9 (2008): 967--991
Jianqing Fan and Qiwei Yao, Nonlinear Time Series: Nonparametric and Parametric Methods [Has a good treatment of additive autoregression models, among many other topics]
Lukas Meier, Sara van de Geer and Peter Bühlmann, "High-Dimensional Additive Modeling", Annals of Statistics 37 (2009): 3779--3821, arxiv:0806.4115
Pradeep Ravikumar, John Lafferty, Han Liu, Larry Wasserman, "Sparse Additive Models", Journal of the Royal Statistical Society B 71 (2009): 1009--1030, arxiv:0711.4555 [a.k.a. "SpAM"]
Simon N. Wood, "Stable and efficient multiple smoothing parameter estimation for generalized additive models", Journal of the American Statistical Association 99 (2004): 673--686 [PDF reprint via Prof. Wood]

Mordecai Ezekiel, "A Method of Handling Curvilinear Correlation for Any Number of Variables", Journal of the American Statistical Association 19 (1924): 431--453 [JSTOR. The oldest publication I have found introducing and advocating additive, as opposed to linear, regression models. The estimation procedure Ezekiel uses is even almost the same as modern "back-fitting", but will get into trouble with correlated input variables.]

The chapter on additive models in Advanced Data Analysis from an Elementary Point of View

Chun-Hao Chang, Sarah Tan, Ben Lengerich, Anna Goldenberg, Rich Caruana, "How Interpretable and Trustworthy are GAMs?", arxiv:2006.06466
Christophe Chesneau, Jalal Fadili, Bertrand Maillot, "Adaptive estimation of an additive regression function from weakly dependent data", arxiv:1111.3994
Andreas Christmann and Robert Hable, "Support vector machines for additive models: consistency and robustness", arxiv:1007.4062
Elise Dusseldorp, Claudio Conversano, and Bart Jan Van Os, "Combining an Additive and Tree-Based Regression Model Simultaneously: STIMA", Journal of Computational and Graphical Statistics (2010) forthcoming
Karl Gregory, Enno Mammen, Martin Wahl, "Statistical inference in sparse high-dimensional additive models", Annals of Statistics 49 (2021): 1514--1536, arxiv:1603.07632
Hong Gu, Toby Kenney, Mu Zhu, "Partial Generalized Additive Models: An Information-Theoretic Approach for Dealing With Concurvity and Selecting Variables", Journal of Computational and Graphical Statistics 19 (2010): 531--551
Benjamin Guedj, Pierre Alquier, "PAC-Bayesian Estimation and Prediction in Sparse Additive Models", arxiv:1208.1211
Jian Huang, Joel L. Horowitz, and Fengrong Wei, "Variable selection in nonparametric additive models", Annals of Statistics 38 (2010): 2282--2313
Jiancheng Jiang, Yingying Fan and Jianqing Fan, "Estimation in additive models with highly or nonhighly correlated covariates", Annals of Statistics 38 (2010): 1403--1432, arxiv:1010.0320
Randy C. S. Lai, Hsin-Cheng Huang, and Thomas C. M. Lee, "Fixed and random effects selection in nonparametric additive mixed models", Electronic Journal of Statistics 6 (2012): 810--842
Andreas Mayr, Nora Fenske, Benjamin Hofner, Thomas Kneib, Matthias Schmid, "Generalized additive models for location, scale and shape for high dimensional data: A flexible approach based on boosting", Journal of the Royal Statistical Society C forthcoming
Juhyun Park and Burkhardt Seifert, "Local additive estimation", Journal of the Royal Statistical Society B 72 (2010): 171--191, arxiv:0806.0612
Veeranjaneyulu Sadhanala and Ryan J. Tibshirani, "Additive models with trend filtering", Annals of Statistics 47 (2019): 3032--3068
Simon N. Wood, "Fast stable direct fitting and smoothness selection for Generalized Additive Models", arxiv:0709.3906
Yingcun Xia, "A Note on the backfitting estimation of additive models", arxiv:0903.3470
Lan Xue, Annie Qu, Jianhui Zhou, "Consistent Model Selection for Marginal Generalized Additive Model for Correlated Data", Journal of the American Statistical Association forthcoming
Takuma Yoshida, Kanta Naito, "Asymptotics for penalized splines in generalized additive models", arxiv:1208.3920
Kyusang Yu, Byeong U. Park, Enno Mammen, "Smooth backfitting in generalized additive models", Annals of Statistics 36 (2008): 228--260, arxiv:0803.1922