## Independence Tests, Conditional Independence Tests, Measures of Dependence and Conditional Dependence

*17 Jul 2022 15:10*

My interest in graphical models and Markov models make me very interested in the following related problems:

- Given samples of two (possible high-dimensional) random variables \( X \) and \( Y \), are they independent?
- Given samples of
*three*random variables \( X \), \( Y \) and \( Z \), are \( X \) and \( Y \) conditionally independent given \( Z \)? - How can we reliably calculate a measure of dependence from sample values?
- Ditto for a measure of conditional dependence.

See also: Density Estimation; Entropy Estimation (mutual information being a fine measure of dependence); Statistics; Two Sample Tests (since if \( X|Y=y_1 \) has a different distribution than \(X | Y=y_2 \), \( X \) and \( Y \) are dependent)

- Recommended:
- Thomas B. Berrett, Richard J. Samworth, "Nonparametric independence testing via mutual information", arxiv:1711.06642
- Jochen Brocker, "A Lower Bound on Arbitrary \( f \)-Divergences in Terms of the Total Variation" arxiv:0903.1765
- Steve Fienberg, The Analysis of Cross-Classified Categorical Data
- Peter Hall, Jeff Racine and Qi Li, "Cross-Validation and the Estimation of Conditional Probability Densities", Journal of the American Statistical Association
**99**(2004): 1015--1026 [PDF] - Jeffrey D. Hart, Nonparametric Smoothing and Lack-of-Fit Tests [comments]
- Solomon W. Kullback, Information Theory and Statistics
- David Lopez-Paz, Philipp Hennig, Bernhard Schölkopf, "The Randomized Dependence Coefficient", arxiv:1304.7717
- Kun Zhang, Jonas Peters, Dominik Janzing, Bernhard Schölkopf, "Kernel-based Conditional Independence Test and Application in Causal Discovery", arxiv:1202.3775

- Modesty forbids me to recommend:
- Daniel J. McDonald, CRS and Mark Schervish
- "Estimating beta-mixing coefficients", AISTATS 2011
- "Estimating Beta-Mixing Coefficients via Histograms", arxiv:1109.5998

- To read:
- Sophie Achard, "A quadratic measure of dependence", math.ST/0609259
- Shotaro Akaho, "A kernel method for canonical correlation analysis", cs.LG/0609071
- Barry C. Arnold et al., Conditional Specification of Statistical Models
- Thomas B. Berrett, Yi Wang, Rina Foygel Barber, Richard J. Samworth, "The conditional permutation test", arxiv:1807.05405
- Patrice Bertail, Paul Doukhan and Philippe Soulier (eds.), Dependence in Probability and Statistics
- G. Biau and L. Gyorfi, "On the Asymptotic Properties of a
Nonparametric $-Test Statistic of
Homogeneity", IEEE
Transactions on Information Theory
**51**(2005): 3965--3973 - Zhanrui Cai, Runze Li, Yaowu Zhang, "A Distribution Free Conditional Independence Test with Applications to Causal Discovery", Journal of Machine Learning Research
**23**(2022): 85 - Cox and Wermuth, Multivariate Dependencies: Models, Analysis and Interpretation
- Sébastien Da Veiga, "Global Sensitivity Analysis with Dependence Measures", arxiv:1311.2483
- Kenji Fukumizu, Arthur Gretton, Xiaohai Sun, Bernhard Schölkopf, "Kernel Measures of Conditional Dependence", NIPS 2007
- Kenji Fukumizu, Le Song, Arthur Gretton, "Kernel Bayes' rule", arxiv:1009.5736
- Arthur Gretton, László Györfi, "Consistent
Nonparametric Tests of
Independence", Journal
of Machine Learning Research
**11**(2010): 1391--1423 - Tzee-Ming Huang, "Testing conditional independence using
maximal nonlinear conditional correlation", Annals of
Statistics
**38**(2010): 2047--2091 - Gusztáv Morvai and Benjamin Weiss, "Testing stationary processes for independence", Annales de l'Institut Henri Poincaré, Probabilités et Statistiques
**47**(2011): 1219--1225 - Leandro Pardo, Statistical Inference Based on Divergence Measures
- Dino Sejdinovic, Bharath Sriperumbudur, Arthur Gretton, Kenji Fukumizu, "Equivalence of distance-based and RKHS-based statistics in hypothesis testing",
Annals of Statistics
**41**(2013): 2263--2291, arxiv:1207.6076 - Xiaofeng Shao, "A generalized portmanteau test of independence between two stationary time series", arxiv:0810.2276
- Gábor J. Székely and Maria L. Rizzo, "Brownian
Distance
Covariance", Annals of
Applied Statistics
**3**(2009): 1236--1265, arxiv:1010.0297 - Gábor J. Székely and Maria L. Rizzo and Nail K. Bakirov, "Measuring and testing dependence by correlation of distances", Annals of Statistics
**35**(2007): 2769--2794, arxiv:0803.4101 - Olivier Thas, Comparing Distributions [mostly about goodness-of-fit tests]