Quantitative and qualitative initial validation of the accountable talk questionnaire

Abstract

Accountable Talk (AT) has been extensively studied as a tool for improving argumentation and respectful discourse. While several research tools exist for evaluating AT, as of yet there no self-report assessment tool to measure AT directly, on a large-scale basis, and without significant costs. The aim of the current study was to develop and provide initial validation for a self-report AT questionnaire (ATQ).One hundred students aged 11–12 years participated in the study. Exploratory Factor Analysis, content validity, and test- retest reliability were assessed. 50 students were also randomly assigned to the qualitative part of the study. These students were recorded while discussing in small groups a topic provided by the experimenter (animal research). The qualitative data was coded and then correlated with the quantitative data obtained from the self-report questionnaire.The results indicated that a 12-items questionnaire can reliably assess three separate and independent qualities of AT: accountability to a learning community, accountability to rigorous thinking, and accountability to accurate knowledge. The reliability of the ATQ was high with α = 0.80. The Test-retest reliability was assessed at two time points separated by a 3-week interval with Pearson correlation. Excellent correlations (r > 0.98) between the ATQ scores were found. The correlation coefficients between the three components of the self-reported ATQ and the observed data obtained from the group discussions were significant positive medium-high.We have shown that a concise 12-item questionnaire can assess the three main components of Accountable Talk within the framework of respectful discourse. The questionnaire showed good reliability and structural validity, with weak correlations between sub-topics supporting the distinction between different aspects of Accountable Talk. We suggest that the ATQ can be used to evaluate the effectiveness of intervention programs aiming to improve students' acquisition of Accountable Talk skills.