We Have Met the Enemy and He Is Us

Numerous authors have demonstrated that when multiple institutional review boards (IRBs) review the same protocol, the results are quite variable (Dilts et al. 2009; Dziak et al. 2005; Green et al. 2006; Greene et al. 2006; Helfand et al. 2009; Kimberly et al. 2006; Mansbach et al. 2007; McWilliams et al. 2003; Silverman et al. 2001; Stair et al. 2001; Stark et al. 2010). However, the federal regulations governing the conduct of human subjects research are gray, by either design or accident, and it is therefore to be expected that reasonable people with differing interpretations will reach disparate conclusions about whether or not a given study meets the criteria for approval (Sayers 2007). Klitzman (2011) provides anecdotal evidence that lends credence to another aspect of IRB variability long known to clinical investigators, namely, that there are differences between the IRBs within institutions. The Klitzman paper is fundamentally based on anecdotes. There is no reason why clinical research into IRB decision making shouldn’t be held to the same standards as those for other types of clinical research. As Halpern (2005) advocated recently, there is a need for evidence-based bioethics to inform decisions. From an epidemiologist’s perspective, large randomized clinical trials rank highest in strength of evidence, followed by smaller clinical trials, cohort studies, case-control studies, and then anecdotal evidence at the bottom. Depending on one’s perspective, meta-analyses combining the results of several trials would be at or near the top of the hierarchy of evidence. Even when a study has a high degree of internal validity, the results from a study may not be generalizable to one’s own situation, patient population, and institution. Given the low position within the hierarchy of evidence occupied by this current report, the results and conclusions reported by Klitzman should be viewed with caution and skepticism. If cause and effect are beyond the realm of the anecdotal study design, then why bother with this type of research? Not all research needs to involve confirmatory designs with formal hypothesis testing. Other exploratory designs are valuable when aimed at generating hypotheses and directions for future research. The power of this paper, then, is not in the amalgamation of the results into a