Optimization of the Preprocessing Method for Edge Detection on Overlapping Cells at PAP Smear Images

The complexity of the cell structure and high overlap cause poor image contrast. Complex imaging factors can make automatic visual interpretation more difficult. Segmentation separates a digital image into different parts with homogeneous attributes so that different areas have different features. The challenges faced in performing nucleus segmentation on Pap Smear (PS) images are poor contrast, the presence of neutrophils, and uneven staining of overlapping cells. This research was conducted to improve image quality in identifying the nucleus accurately. The method used is the Polynomial Contrast Enhancement (PCE) model as an approach to preprocessing. This method functions to change the contrast of the Pap smear image against the overlapping cells so that it becomes a significant contrast in detecting the edge of the nucleus object. The detection process uses the Robert and Prewitt edge detection method to test the identification of the nucleus object on 797 PS Repository images of the University of Nusa Mandiri (RepomedUNM). The accuracy result obtained is 86.8%. Comparing Robert's edge detection and Prewitt's edge detection shows that the PCE approach as a filter method can overcome color contrast problems and detect more accurately. The difficulty in detecting the nucleus from the PS image against the overlapping cells can be solved. This method can distinguish overlapping cells from their core during testing, thus becoming a reference in identifying cells with improved accuracy and testing larger data sets.