Localizing a needle tip using 2D microscope images and detecting vertical approach of a needle based on focus measures for intracellular microneedle insertion

Microneedle insertion is useful for elucidating the processes in living cells and their organelles. However, mechanical puncture by the needle causes traumatic damage to the cell. As a less invasive process, we have developed intracellular needle insertion using femtosecond laser cell ablation. To quickly and precisely position the spot ablated by the laser, we accurately located the needle tip in a 2D image plane. When the needle approaches the cell surface, the only accessible information is the microscope image; the vertical approach of the needle is unknown and must be detected. To this end, we propose an image process that integrates needle recognition, needle-tip localization, image focus measure, and vertical approach detection. The proposed image process was tested in 15 experimental sessions at various locations. The needle tip was reasonably and consistently positioned in the image, and the vertical approach was detected within the safe and plausible ranges.