Medical Expert Systems: Issues of Validation, Evaluation and Judgment

Medical expert systems often present quite difficult problems of validation and evaluation because their knowledge bases typically contain many implicitly represented value judgments, which may not be easily separated from statements of uncertainty and fact, as in a formal decision theoretic rnodel. Thus, it is the very flexibility of knowledge representation allowed by expert system frameworks which makes the job of knowledge validation and system performance evaluation more complex than for simpler decision support systems. And, because of the ethical and legal constraints on medicail knowledge utilization and testing, it has been difficult to produce systematic mothodologies for evaluating the performance of these systems. It is therefore not surprising that they have not been as widely adopted as expert systems in other domains which have been derived from the pioneering work in medical systems. While new methods of knowledge base refinement hold out the promise for technical improvements, issues of responsibility for the application, testing and evaluation of expert systems will continue to provide fertile ground for discussion of the extra-logical elements of decision making.