Discussion to the paper ‘Robust Distance Covariance’

Abstract

We would like to thank S. Leyder, J. Raymaekers, and P. Rousseeuw for their insightful and well-written article ‘Robust Distance Covariance’. The paper provides a clear and comprehensive overview of distance covariance and related concepts and derives their influence functions and breakdown points. It is shown that, depending on parameter choices, classical distance covariance can exhibit a degree of robustness, although it still falls somewhat short in this regard.

To address this limitation, the authors introduce a novel robust version of distance covariance. By transforming the data into a higher-dimensional bounded domain, their method achieves a high breakdown point while preserving the desirable property that a zero value of distance covariance implies independence between the random elements involved.

Historically, the robust statistics literature has paid limited attention to the properties of statistical methods in settings where independence is the central concept. This may be attributed to the dominance of elliptical models, where independence is typically not a relevant notion. However, the growing popularity of models such as the independent component model (see Nordhausen and Oja 2018) has increasingly motivated the investigation of robust estimation techniques in scenarios where independence plays a critical role (Nordhausen and Tyler 2015).

These investigations have revealed that many scatter functionals fail to return zeros at the appropriate entries when components are independent, especially in the presence of asymmetry. As shown in Oja et al. (2006) and Nordhausen and Tyler (2015), a fruitful strategy is to symmetrise the data and operate on pairwise differences—a principle that is also inherent in the concept of distance covariance, as further highlighted in Raymaekers and Rousseeuw (2025).

In this discussion, we are particularly interested in the application of (robust) generalized distance covariance in multivariate contexts, extending beyond the primarily bivariate focus of S. Leyder, J. Raymaekers, and P. Rousseeuw's article. At first glance, it may seem unusual that, regardless of the dimensionality of the vectors considered, (robust) generalized distance covariance returns a scalar rather than a matrix. From a multivariate perspective, this naturally invites consideration of the method within the framework of projection pursuit.

The concept of projection pursuit (PP) dates back several decades, with the term itself introduced by Friedman and Tukey (1974). The fundamental idea is to apply a (typically univariate) measure of ‘information’ to multivariate data by identifying linear combinations of the variables that maximise this measure. The resulting directions are considered the most informative components of the data.

A wide variety of information measures have been proposed, and many well-known statistical methods can be interpreted within the projection pursuit framework. For instance, principal component analysis (PCA) uses variation as its projection index, seeking directions of maximum variability in the data.

Similarly, independent component analysis (ICA) can be viewed as a PP approach where projections are chosen to maximise measures of non-Gaussianity.

Since more than one informative direction is usually of interest, constraints (such as orthogonality or statistical independence) are typically imposed to facilitate the extraction of subsequent components. Building on this idea, Radojičić et al. (2020) explore how squared measures of scale can be utilised in the context of ICA.

For symmetric distributions, dispersion is interpreted in terms of the distance of a random variable $X$ from its central value $\mu$, specifically through the magnitude of $\mid X-\mu \mid$, and is closely related to the ‘peakedness’ ordering introduced by Birnbaum (1948). Perhaps one of the most used measures of dispersion is a standard deviation.

For non-symmetric distributions, perhaps a more meaningful measure is that of spread. A non-negative functional ${\tau }_2$ satisfying (1) is according to Bickel and Lehmann (2011b) a measure of spread if, in addition, $F^{-1}\left(b\right)-F^{-1}\left(a\right)\le G^{-1}\left(b\right)-G^{-1}\left(a\right)$ implies that ${\tau }_2\left(X\right)\le {\tau }_2\left(Y\right)$ for every $0<a\le b<1$, where $F^{-1}$ and $G^{-1}$ are the right-continuous inverses of the cumulative distribution functions of $X$ and $Y$, respectively. We will write $X\preccurlyeq Y$. Thus, for symmetric distributions, the spread measure is also a measure of dispersion. The converse, however, does not hold. (Edelmann et al. 2020) show that if ${\tau }_1\left(X\right)$ is a measure of dispersion in the upper sense, then ${\tau }_1\left(X-X^{\prime }\right)$ is a measure of spread, where $X^{\prime }$ is an independent copy of $X$. One example of a spread measure is the standard deviation. Edelmann et al. (2020) additionally showed that distance standard deviation $\text{dStd}\left(X\right)$ is indeed a measure of spread.

Both dispersion and the spread measures are subclasses of the collection of all non-negative functionals $\tau$ satisfying (1). Due to the equivariance of such measures to the change in scale, we will refer to those as scale measures. It is straightforward to see that for $0<\alpha <2$, generalized distance standard deviation $\text{dStd}{\left(X,\alpha \right)}^{1/2}=\sqrt{\text{dCov}{\left(X,X;\alpha \right)}^{1/\alpha }}$ is a scale measure. However, if it is a spread measure in the sense of Bickel and Lehmann (2011b) is not that evident. We thus focus on $\alpha =1$, that is, $\text{dStd}\left(X\right)$ in the further discussion.

PCA finds the most ‘informative’ directions by maximizing variability, but classical PCA relies on standard deviation, which is sensitive to outliers and requires finite second moments. This makes it less reliable for heavy-tailed data. In Li and Chen (1985) and Croux and Ruiz-Gazen (2005) robust measures of scale are then suggested to replace the standard deviation in PP to yield robust PCA variants. The question is then if the robust variant of $\text{dStd}$ can still be used for this purpose and how the spread of the non-linear data transformation like biloop connects to the spread of the original data.

S. Leyder, J. Raymaekers, and P. Rousseeuw discuss in the robust context mainly $\text{dCor}$ and indicate that the ${\Psi }_{\infty }$ is not actually applied directly to the random variable $X$ but rather to $X^{\mathrm{st}}=\left(X-t_x\right)/m_X$ where $t_X$ denotes the median of $X$ and $m_X$ the corresponding median absolute deviation. As $\text{dStd}$ is a measure of spread we suggest to define $\mathrm{rob}\_\text{dStd}\left(X\right)$ as $\mathrm{rob}\_\text{dStd}\left(X\right)=\text{dStd}\left(m_X{\Psi }_{\infty }\left(X^{\mathrm{st}}\right)\right)$ to ensure that $\mathrm{rob}\_\text{dStd}\left(\mathrm{aX}+b\right)=\mid a\mid \mathrm{rob}\_\text{dStd}\left(X\right)$ for all $a,b\in ℝ$.

As $\text{dStd}$ and $\mathrm{rob}\_\text{dStd}$ are now measures of scale, they can also be used as a projection index to obtain a potential robust estimator of principal components, building, for example, on those discussed in Li and Chen (1985) and Croux and Ruiz-Gazen (2005).

As in traditional PCA, the remaining $p-1$ directions are found optimizing (2) subject to the orthogonality constraint to the previous optimizers. Variants of $\text{dStd}$, as reviewed and proposed in the paper, might provide, therefore, an even more robust variant of PCA.

Since both traditional standard deviation and distance standard deviation are spread measures, on a population level, these recover the same components. To illustrate, let $u_{1,\mathrm{sd}}$ be the first traditional principal direction. Then, for any $\parallel v\parallel =1$, $\mathrm{sd}\left(u_{1\mathrm{sd}}^{\prime }X\right)\ge \mathrm{sd}\left(v^{\prime }X\right)$ which is equivalent to $v^{\prime }X\preccurlyeq u_{1,\mathrm{sd}}^{\prime }X\iff \text{dStd}\left(v^{\prime }X\right)\le \text{dStd}\left(u_{1,\mathrm{sd}}^{\prime }X\right)$. Therefore, provided that all the principal components have distinct (distance) standard deviations, the recovery and the ordering of PCs are preserved with $\text{dStd}-\mathrm{PCA}$.

To illustrate this, we apply the PP approach to two different bivariate datasets, each with a sample size of 200. One follows a normal distribution, while the other is a linear mixture of two independent skewed components. Both datasets have the same expected value and the same covariance matrix. In addition, we added 5 outliers to each dataset to examine the impact when using sd, $\text{dStd}$, and $\mathrm{rob}\_\text{dStd}$ as scale measures. Figure 1 shows, in four panels, the two settings—once without contamination and once with contamination—and visualizes the direction of the first principal component. Clearly, in the uncontaminated case, all methods point in quite similar directions, whereas in the contaminated case, only $\mathrm{rob}\_\text{dStd}$ appears unaffected.

Often, the reasoning in PP-based ICA is to search for directions for which the projection is as far as possible from the Gaussianity (in some sense). Edelmann et al. (2020) show that for independent random variables $X$ and $Y$, such that $X$ has log-concave density $\text{dStd}\left(X+Y\right)\ge \text{dStd}\left(Y\right).$ As the Gaussian distribution has log-concave density, the upper result could be interpreted that with respect to distance covariance, it is not a very interesting distribution. The same holds for the rest of bell-shaped symmetric distributions, which are in ICA, often taken to be noise. Therefore, adding (in any direction) even a small amount of signal increases $\text{dStd}$ when compared with pure noise.

Above, we used the property that $\text{dCov}$ returns always a single value and how the information of scale provided by it can be used in the context of multivariate analysis. Summarizing the interrelations between multiple random variables into a single value, as done by distance covariance, may not always capture the full picture. Already Edelmann et al. (2020) suggested addressing this limitation by constructing, for a $p$-variate random vector $x$, a $p\times p$ matrix $\Gamma \left(x\right)$, where the $\left(i,j\right)$th entry is given by $\text{dCov}\left(x_i,x_j\right)$ for $i,j=1,\dots ,p$, and proposed this as a topic for future research. To our knowledge, this idea has not been further developed, but we believe it could be of interest—especially in light of the new possibility of robustifying the individual elements.

The work of KN was partly supported by HiTEc COST Action (CA21163) and by the Research Council of Finland (363261). The work of UR was supported by the Austrian Science Fund (FWF, 10.55776/I5799).