Model based segmentation of nuclei

A new approach for segmentation of nuclei observed with an epi-fluorescence microscope is presented. The technique is model based and uses local feature activities such as step-edge segments, roof-edge segments, and concave corners to construct a set of initial hypotheses. These local feature activities are extracted using either local or global operators to form a possible set of hypotheses. Each hypothesis is expressed as a hyperquadric for better stability, compactness, and error handling. The search space is expressed as an assignment matrix with an appropriate cost function to ensure local adjacency, and global consistency. Each possible configuration of a set of nuclei defines a path, and the path with the least error corresponds to best representation. This result is then presented to an operator who verifies and eliminates a small number of errors.