Response to “Importance of method assumptions”

Abstract

TO THE EDITOR: We read with interest the recent letter concerning our self-criticism [(1)] of the nonlinear Mendelian randomization (MR) methodology that we applied in an earlier publication in this journal [(2)]. In a constructive critique, the authors challenge the suitability of assigned sex as a negative control outcome under the definition that a valid negative control outcome should not be plausibly related to the exposure (here, body mass index [BMI]) and that these conditions can be tested within an analysis framework free of confounding and selection bias.

This is a reasonable point of consideration and one that justifiably cross-examines our use of the term “negative control.” However, this flags a more general issue—that our use of this term may not have been helpful from a formal point of view. Sadly, we fear that, again, the likely complications of nonlinear MR as described in our recent Perspective have been missed—that there are fundamental difficulties in the application of nonlinear MR analyses and that current methods (including the doubly ranked approach) are unable to avoid the generation of spurious associations.

We concede that the use of the UK Biobank as a population sample is potentially imperfect (mostly due to the biased, volunteer-based sampling frame) [(3)] for the deployment of a pure negative control analysis, as it will be for many other forms of both observational and applied epidemiological investigations. We have not provided an exhaustive simulation of the likely sources of possible bias to this extent, nor have we undertaken the same analysis in multiple independent studies (of different design) in efforts to quantify such flaws. However, what is important is that the analyses of both primary outcome and negative control (here, assigned sex) were undertaken in the same study and will suffer many of the same problems while remaining implausibly related; BMI may well be related to assigned sex, but BMI does not cause the latter. Importantly, the magnitude of selection bias within this sample and the specific question is unlikely to be large enough to explain these results, and, even if BMI was introducing the selection bias, this would influence the overall MR results as much as was seen using nonlinear methods and most likely not in a different and incomprehensible manner [(4, 5)].

There are a series of other potential arguments that would have been more well placed targeting our own use of the phrase negative control. For example, we may well expect a variable claimed to be an “instrument” for BMI that is derived from a polygenic risk score [(4)] to behave differently across strata of BMI (however formed), owing to the dimorphic characteristics of the underpinning genome-wide association study results. This is a potential biological flaw to the analysis and indeed one in which the differences in stratum-specific estimates would be induced by the nature of relationship between instrument and negative control. However, even this, with anticipated properties, would not generate the clearly flawed results generated in the illustrative analyses. Critically, the undertaking of nonlinear MR (even with new methods available that continue to generate questionable results) [(1, 6)] remains unreliable and is likely influenced by a host of features manifest with differing impact across studies (e.g., genetic architecture of exposures, outcome of interest, discovery genetic studies, distribution of exposure and relationship between exposure and outcome).

We welcome the authors' overall point regarding the need for the use of triangulation to ameliorate, or at least contextualize, inferential difficulties encountered in any study design. However, we are still unconvinced that the refinements to analyses or wording are such to correct the problems flagged in our exemplar analysis from 2018 or to rescue existing methods attempting to undertake nonlinear MR.

The authors declared no conflict of interest.