Analyzing large free-response qualitative data sets — a novel quantitative-qualitative hybrid approach

Abstract

Qualitative analysis tends to be unwieldy for large data sets yet is an indispensable tool for understanding how and why phenomena occur. Consequently, the goal of this study was to develop a method that is credible yet economical for large, specific, qualitative data sets. The strength of our hybrid, qualitative-quantitative method comes from using automated text analysis techniques to focus resource-intensive coding efforts on a small, carefully selected subset of data. This paper details the hybrid method as applied to a previously analyzed set of free-response data and argues for the methodpsilas validity by comparing results from the hybrid analysis with the previous traditional qualitatively analyzed method. With this data set, the hybrid method yielded comparable results with substantially less manual coding and in less than a third of the time required for the original analysis method. This hybrid analysis provides a more economical alternative for a ldquocoarse-cutrdquo qualitative analysis and observation of long-term trends, providing insight to practitioners, assessors, and researchers ranging from individual course evaluations to large-scale studies. Short, focused, open-ended survey questions are good candidates for this type of analysis.