Analytic prediction of medical document retrieval system performance

Abstract

The performance of medical information retrieval systems is measured using historical data or predicted using formal probabilistic methods derived from artificial intelligence and statistical decision theoretic considerations. Technique have been described that assist the searcher with a query or information need by providing graphs showing the quality of past retrieval performance for that specific query, as well as expected future performance. Documents or text fragments (from a hypertext system) are ranked for possible presentation to the searcher based on the document of fragment's odds of relevance. The expected performance is computed from knowledge gained from relevance judgments provided by the searcher about the quality of the retrieved documents, as well as any system knowledge available about possible initial values of parameters of distributions describing the occurrence of features of relevance and all text fragments. The individual documents do not need to be examined to predict performance.<>