Performance characterization of vision algorithms

In order to design vision systems which work, a sound engineering methodology must be utilized. In the systems engineering approach, a complex system is divided into simple subsystems and from the input/output characteristics of each subsystem, the input/output characteristics of the total system can be determined. Machine vision systems are complex, and they are composed of different algorithms applied in sequence. Determination of the performance of a total machine vision system is possible if the performance of each of the subpieces, i.e. the algorithms, is given. The problem, however, is that for most algorithms, there is no performance characterization which has been established and published in the research literature. Performance characterization has to do with establishing the correspondence of the random variations and imperfections which the algorithm produces on the output data caused by the random variations and imperfections of the input data. This paper illustrates how random perturbation models and propagation of random errors can be set up for a vision algorithm involving edge detection, edge linking, arc segmentation, and line fitting. The paper also discusses important dimensions that must be included in the performance characterization of any vision module performing a parametric estimation such as object pose, curve fit, or edge orientation estimation. Finally, we outline a general parametric model having three components: a relational model; a noise model; and a computational estimation model.