Penalized Bayesian methods for product ranking using both positive and negative references.

Abstract

Product ranking according to pre-specified criteria is essential for developing new technologies, allowing identification of more preferable candidates for further development. Such ranking often builds on the results of a network meta-analysis, where the relative or absolute performances of the various products are synthesized across multiple clinical studies, each of which considered only a subset of the products. Ranking involving both a negative and a positive reference enables the scientist to directly compare tested products against known benchmarks. Here, more preferable candidates are those products that approach the positive reference while remaining distant from the negative reference. We provide a new metric to quantify this multivariate distance following Bayesian meta-analysis. Our method does not simply rely on point estimates to perform the comparisons, but also accounts for their uncertainties via their posterior distributions. For each product, posterior probabilities of being comparable to the positive reference are computed, and subsequently penalized by the posterior probability of performing worse than the negative reference. Each product is then compared to a hypothetical product about which we have no knowledge, as captured by a uniform distribution. The result is a prospective metric that is directly interpretable as the improvement of any product beyond this state of ignorance. We illustrate our approach using a case study, in which the goal is to rank 16 antiperspirant products. Here, the FDA-recommended summary statistic (a measure of the relative sweat reduction between each product and no treatment) intrinsically features both positive and negative references. We then offer a brief simulation study to check our metric's performance in less complex, idealized settings where the true ranking is known. Our results indicate that our Bayesian approach is a novel and useful addition to the statistical ranking toolkit.