Towards multimodal visualization of esophageal motility: fusion of manometry, impedance, and videofluoroscopic image sequences.

Purpose: Dysphagia is the inability or difficulty to swallow normally. Standard procedures for diagnosing the exact disease are, among others, X-ray videofluoroscopy, manometry and impedance examinations, usually performed consecutively. In order to gain more insights, ongoing research is aiming to collect these different modalities at the same time, with the goal to present them in a joint visualization. One idea to create a combined view is the projection of the manometry and impedance values onto the right location in the X-ray images. This requires to identify the exact sensor locations in the images.

Methods: This work gives an overview of the challenges associated with the sensor detection task and proposes a robust approach to detect the sensors in X-ray image sequences, ultimately allowing to project the manometry and impedance values onto the right location in the images.

Results: The developed sensor detection approach is evaluated on a total of 14 sequences from different patients, achieving a F1-score of 86.36%. To demonstrate the robustness of the approach, another study is performed by adding different levels of noise to the images, with the performance of our sensor detection method only slightly decreasing in these scenarios. This robust sensor detection provides the basis to accurately project manometry and impedance values onto the images, allowing to create a multimodal visualization of the swallow process. The resulting visualizations are evaluated qualitatively by domain experts, indicating a great benefit of this proposed fused visualization approach.

Conclusion: Using our preprocessing and sensor detection method, we show that the sensor detection task can be successfully approached with high accuracy. This allows to create a novel, multimodal visualization of esophageal motility, helping to provide more insights into swallow disorders of patients.