Independence Tests, Conditional Independence Tests, Measures of Dependence and Conditional Dependence
17 Jul 2022 15:10
My interest in graphical models and Markov models make me very interested in the following related problems:
- Given samples of two (possible high-dimensional) random variables \( X \) and \( Y \), are they independent?
- Given samples of three random variables \( X \), \( Y \) and \( Z \), are \( X \) and \( Y \) conditionally independent given \( Z \)?
- How can we reliably calculate a measure of dependence from sample values?
- Ditto for a measure of conditional dependence.
See also: Density Estimation; Entropy Estimation (mutual information being a fine measure of dependence); Statistics; Two Sample Tests (since if \( X|Y=y_1 \) has a different distribution than \(X | Y=y_2 \), \( X \) and \( Y \) are dependent)
- Recommended:
- Thomas B. Berrett, Richard J. Samworth, "Nonparametric independence testing via mutual information", arxiv:1711.06642
- Jochen Brocker, "A Lower Bound on Arbitrary \( f \)-Divergences in Terms of the Total Variation" arxiv:0903.1765
- Steve Fienberg, The Analysis of Cross-Classified Categorical Data
- Peter Hall, Jeff Racine and Qi Li, "Cross-Validation and the Estimation of Conditional Probability Densities", Journal of the American Statistical Association 99 (2004): 1015--1026 [PDF]
- Jeffrey D. Hart, Nonparametric Smoothing and Lack-of-Fit Tests [comments]
- Solomon W. Kullback, Information Theory and Statistics
- David Lopez-Paz, Philipp Hennig, Bernhard Schölkopf, "The Randomized Dependence Coefficient", arxiv:1304.7717
- Kun Zhang, Jonas Peters, Dominik Janzing, Bernhard Schölkopf, "Kernel-based Conditional Independence Test and Application in Causal Discovery", arxiv:1202.3775
- Modesty forbids me to recommend:
- Daniel J. McDonald, CRS and Mark Schervish
- "Estimating beta-mixing coefficients", AISTATS 2011
- "Estimating Beta-Mixing Coefficients via Histograms", arxiv:1109.5998
- To read:
- Sophie Achard, "A quadratic measure of dependence", math.ST/0609259
- Shotaro Akaho, "A kernel method for canonical correlation analysis", cs.LG/0609071
- Barry C. Arnold et al., Conditional Specification of Statistical Models
- Thomas B. Berrett, Yi Wang, Rina Foygel Barber, Richard J. Samworth, "The conditional permutation test", arxiv:1807.05405
- Patrice Bertail, Paul Doukhan and Philippe Soulier (eds.), Dependence in Probability and Statistics
- G. Biau and L. Gyorfi, "On the Asymptotic Properties of a Nonparametric $-Test Statistic of Homogeneity", IEEE Transactions on Information Theory 51 (2005): 3965--3973
- Zhanrui Cai, Runze Li, Yaowu Zhang, "A Distribution Free Conditional Independence Test with Applications to Causal Discovery", Journal of Machine Learning Research 23 (2022): 85
- Cox and Wermuth, Multivariate Dependencies: Models, Analysis and Interpretation
- Sébastien Da Veiga, "Global Sensitivity Analysis with Dependence Measures", arxiv:1311.2483
- Kenji Fukumizu, Arthur Gretton, Xiaohai Sun, Bernhard Schölkopf, "Kernel Measures of Conditional Dependence", NIPS 2007
- Kenji Fukumizu, Le Song, Arthur Gretton, "Kernel Bayes' rule", arxiv:1009.5736
- Arthur Gretton, László Györfi, "Consistent Nonparametric Tests of Independence", Journal of Machine Learning Research 11 (2010): 1391--1423
- Tzee-Ming Huang, "Testing conditional independence using maximal nonlinear conditional correlation", Annals of Statistics 38 (2010): 2047--2091
- Gusztáv Morvai and Benjamin Weiss, "Testing stationary processes for independence", Annales de l'Institut Henri Poincaré, Probabilités et Statistiques 47 (2011): 1219--1225
- Leandro Pardo, Statistical Inference Based on Divergence Measures
- Dino Sejdinovic, Bharath Sriperumbudur, Arthur Gretton, Kenji Fukumizu, "Equivalence of distance-based and RKHS-based statistics in hypothesis testing", Annals of Statistics 41 (2013): 2263--2291, arxiv:1207.6076
- Xiaofeng Shao, "A generalized portmanteau test of independence between two stationary time series", arxiv:0810.2276
- Gábor J. Székely and Maria L. Rizzo, "Brownian Distance Covariance", Annals of Applied Statistics 3 (2009): 1236--1265, arxiv:1010.0297
- Gábor J. Székely and Maria L. Rizzo and Nail K. Bakirov, "Measuring and testing dependence by correlation of distances", Annals of Statistics 35 (2007): 2769--2794, arxiv:0803.4101
- Olivier Thas, Comparing Distributions [mostly about goodness-of-fit tests]
