Quantifying the completeness and reliability of visual source extraction: An examination of eight thousand data cubes by eye

Context. Source extraction in HI radio surveys is still often performed using visual (by-eye) inspection, but the efficacy of the procedure lacks rigorous quantitative assessment due its laborious nature. Thus, algorithmic methods are often preferred due to their repeatable results and speed.Aims. This work attempts to quantitatively assess the completeness and reliability of visual source extraction by using a suitably large sample of artificial sources and a comparatively rapid source extraction tool and to compare the results with those from automatic techniques.Methods. A dedicated source extraction tool was modified to significantly reduce the cataloguing speed. I injected 4232 sources into a total of 8500 emission-free data cubes, with at most one source per cube. The sources covered a wide range of signal-to-noise values and velocity widths. I blindly searched all cubes for the sources, measuring the completeness and reliability for pairs of signal-to-noise and line width values. Smaller control tests were performed to account for the possible biases in the search, which gave results in good agreement with the main experiment. I also searched cubes injected with artificial sources using algorithmic extractors and compared these results with a set of catalogues independently reported from real observational data, which were searched with different automatic methods.Results. I find that the results of visual extraction follow a tight relation between integrated signal-to-noise and completeness. Visual extraction compares favourably in efficacy with the algorithmic methods, tending to recover a higher fraction of fainter sources.Conclusions. Visual source extraction can be a surprisingly rapid procedure that yields higher completeness levels than automatic techniques, giving predictable and quantifiable results that are not strongly subject to the whims of the observer. Regarding the recovery of the faintest features, algorithmic extractors can be competitive with visual inspection but do not yet outperform it, though their advantage in speed can be a significant compensating factor.