Discussion on the SJS invited paper by Sander Greenland Divergence vs. Decision P$$ P $$‐values: A Distinction worth making in theory and keeping in Practice

Abstract

It is shown in the cited paper written by Michael Lavine and in several others works that the p -value of the test-statistics is not a consistent measure of evidence in the context of testing alternative hypothesis. As Sander Greenland points out, these should not be confused with the p -values of realized goodness of-fit-test statistics. Goodness-of-fit tests are useful sanity checks in order to decide, (and there would be also a decision to be taken there !) whether our model and/or assumptions are compatible with the available data, or we need to take a step back and look for what could be wrong. Greenland argues that although using the p -value in a goodness of fit test violates the likelihood principle, these p -values do not suffer the inconsistencies of p -values arising from testing alternative hypothesis, and can be used as a measure of evidence against the model, (following Popper, scientific theories and models can be only falsified and never verified), on an absolute scale in the [ 0 , 1 ] range, where a zero left-sided p -value corresponds to perfect fit, and every p -value strictly greater than zero should be considered as negative evidence of some level. I find this peculiar, unless we would be dealing only with fully deterministic models. Fitting the data is not