IRB Inconsistency: Be Careful What You Wish For

Abstract

Robert Klitzman’s report (2011) of his study of discrepancies among institutional review boards (IRBs) reminds me of the joke about two rabbis who met in the desert. After extensive arguing, unable to agree on anything, they decided they not only had to build separate temples, but a third temple that neither would set foot in. Differences and inconsistency are inherent in human activity (and canine, feline, etc.). We should not be surprised when two humans differ in how they apply a regulation to a particular case, whether it be as simple as a traffic light rule, as complex as the tax code, or the federal rules on human subjects research. Similarly, disagreement among groups of individuals, such as IRBs, whether within one institution or across multiple institutions, should be expected. Even an individual member may take inconsistent positions on the same issue arising in different protocols. As Klitzman notes, these disagreements among IRBs are due to numerous factors, including varying knowledge about scientific issues, personal biases, political factors, personalities, small-group dynamics, leadership styles, community values, and so on. His study suggests that differences in community values may not be the most important or common reason for these intergroup differences. As one of Klitzman’s respondents noted, this is not different from clinical medicine, where equally qualified physicians commonly have widely different views of diagnosis, prognosis, and treatment of the same patient. In one study, not only did a group of experienced, qualified radiologists disagree about the findings in a group of chest x-rays; when the same group was asked to read the same x-rays a year later, there was no correlation within each individual’s interpretations from one year to the next.