AppRaise: Software for Quantifying Evidence Uncertainty in Systematic Reviews Using a Posterior Mixture Model

Abstract

Rationale

Systematic reviews are essential for evidence-based healthcare decision-making. While it is relatively straightforward to quantitatively assess random errors in systematic reviews, as these are typically reported in primary studies, the assessment of biases often remains narrative. Primary studies seldom provide quantitative estimates of biases and their uncertainties, resulting in systematic reviews rarely including such measurements. Additionally, evidence appraisers often face time constraints and technical challenges that prevent them from conducting quantitative bias assessments themselves. Given that multiple biases and random errors collectively skew the point estimate from the truth, it is important to incorporate comprehensive quantitative methods of uncertainty in systematic reviews. These methods should integrate random errors and biases into a unified measure of uncertainty and be easily accessible to evidence appraisers, preferably through user-friendly software.

Aims and Objectives

To address this need, we propose a posterior mixture model and introduce AppRaise, a free, web-based interactive software designed to implement this approach.

Methods

We showcase its application through a health technology assessment (HTA) report on the effectiveness of continuous glucose monitoring in reducing A1c levels among individuals with type 1 diabetes.

Results

Applying the AppRaise software to the HTA report revealed a high level of certainty (86% probability) that continuous glucose monitoring would, on average, result in a reduction in A1c levels compared with self-monitoring of blood glucose among Ontarians with type 1 diabetes. These findings were similar to other quantitative bias-adjusted approaches in systematic reviews.

Conclusion

AppRaise can be utilized as a standalone tool or as a complement to validate the quality of evidence assessed using qualitative-based scoring methods. This approach is also useful for assessing the sensitivity of parameter estimates to potential biases introduced by primary studies.