Avoiding damned lies: understanding statistical ideas

Abstract

Many researchers and practitioners in HCI will at some time or another need to use or interpret experimental statistics. However, the correct use of statistics involves a combination of mathematics and practical know-how. Often those who have studied an introductory statistics course have learnt how to perform the requisite mathematical manipulation, but not the meaning of the resulting numbers. This tutorial aims to fill in the understanding gap experienced by many who are using statistics, but do not feel ‘on top’ of it. It will focus on the meaning of a few key concepts and some of the common mistakes and fallacies prevalent in the HCI literature. both cases the results were too good to be true. A systematic process had been at work the experimenters had discarded those results which disagreed with their hypothesis. In fact, the results they discarded would have been simply the results of randomness making some experiments run counter to the general trend. This is quite normal and to be expected. So, don’t try to fiddle your results you will be found out!