General Principles of Hypothesis Testing

In Chapter 1, we described an experiment by Barlett (2015) in which he attempted to investigate whether there is a difference in hostility between those who receive insulting or nice online messages by conducting an experiment in which participants received messages that were either insulting or nice and then measuring the participants’ levels of hostility. We presented the results of this experiment at the beginning of Chapter 2. In this chapter, we will apply the concepts discussed in preceding chapters to describe the basic principles for testing statistical hypotheses. To make it easier to see those basic principles, we will assume for the moment that we know the population variances. We will postpone the actual analysis of Barlett’s data until Chapter 7, where we will use estimates of the population variance in the application of Student’s t-test. As we saw in Chapter 1, we start with a research question and generate mutually exclusive and exhaustive experimental hypotheses as possible answers to our research question. Then we design a research study based on our research hypotheses and collect data. By making certain assumptions about the data, we can use a statistical model to assess whether the obtained results reflect real experimental effects or merely random (chance) factors. With the classical statistical model, this assessment is carried out by making assumptions about the shape of the populations from which the data were obtained, setting up statistical hypotheses about the parameters of these populations, and evaluating which hypothesis is best supported by the data. The results of our statistical hypothesis test are then generalized back to our experimental hypotheses to hopefully answer the question originally posed. In this chapter, we will examine the principles involved in testing statistical hypotheses with the classical statistical model, and in Chapter 9, we will do the same with the randomization/ permutation model.