Recording and communicating uncertainty in science: how geologists manage variability in spatial data.

Abstract

All scientists must cope with variability in data to make inferences about the world. However, in observation-based geology, how scientists cope with variability is particularly consequential because it determines what become data in the first place, with observations that are deemed "too variable" potentially being ignored or minimized. Here, across three experiments with 97 geologists, we assess (i) how variability impacts their willingness to turn an observation into data by recording it and their willingness to share data by publishing it, and (ii) whether scientists can make inferences from variable observations and how the accuracy of their inferences is impacted by level of variability. Geologists were presented with arrays of disciplinary data representing the orientation of planar features within a rock formation, where orientation variability was systematically manipulated. Results showed substantial individual differences in criterion tolerance of variability: high-criterion individuals perceived low-to-moderate degrees of variability as more noise than signal and were never willing to publish high variability data (and often not willing to record it), while those with low criterion perceived low-to-moderate degrees of variability as more signal than noise and were always willing to record high variability data (and often publish it). Regardless of tolerance for variability, geologists overall were good at making accurate orientation estimates from variable data, even at the highest levels of variability employed in the study. Together, these results imply there may be situations where scientists avoid recording or publishing variable data, despite being able to draw meaningful conclusions from such data.