Two metrics for quantifying systematic errors in diffraction experiments: systematic errors in the variance of the observed intensities and agreement factor gap.

The increase in the weighted agreement factor due to systematic errors in single-crystal X-ray and neutron diffraction experiments can be quantified precisely, provided the estimated standard uncertainties of the observed intensities, s.u.(I _obs), are sufficiently accurate. The increase in the weighted agreement factor quantifies the 'costs' of the systematic errors. This is achieved by comparison with the lowest possible weighted agreement factor for the specific data set. Application to 314 published data sets from inorganic, metal-organic and organic compounds shows that systematic errors increase the weighted agreement factor by a surprisingly large factor of g = 3.31 (or more) in 50% of the small-molecule data sets from the sample. Examples of twinning, disorder, neglect of bonding densities and low-energy contamination are taken from the literature and examined with respect to the increase in the weighted agreement factor, which is typically less than three. The large value g = 3.31 for the supposedly simple case of rather small molecules, as opposed to macromolecules, is interpreted as a warning sign that there are not only the expected remaining systematic errors, like not-modelled disorder, unrecognized twinning or neglect of bonding electrons or similar errors, but additionally a common systematic error of insufficiently accurate s.u.(I _obs). Inadequate s.u.(I _obs) may not just compromise the model parameters and model parameter errors; they are also a threat to the whole data quality evaluation procedure that relies crucially on adequate s.u.(I _obs).