Ways of Qualitative Coding: A Case Study of Four Strategies for Resolving Disagreements

Abstract

The process of qualitative coding often involves multiple coders coding the same data to ensure reliable codes and a consistent understanding of the codebook. One aspect of qualitative coding includes resolving disagreements, where coders discuss differences in coding to reach a consensus. We conduct a case study to evaluate four strategies of disagreement resolution and understand their impact on the coding process. We find that an open discussion and the n-ary tree metric lead coders to focus more on the disagreement of a particular data instance, whereas kappa values and Code Wizard direct coders to compare code definitions. We discuss opportunities for using different strategies at different stages of the coding process for more effective disagreement resolution.