Concordance of renal tumour assessments by urologists using CT scans versus Hyper-Accuracy 3D Virtual Models for surgical planning: A single-centre multireviewer analysis

Abstract

Robot-assisted partial nephrectomy (RAPN) might represent a surgically demanding procedure that conceals several pitfalls, including proximity to vessels or calyces and lesions not visible upon kidney exposure. Therefore, a comprehensive understanding of key anatomical landmarks is crucial for precise surgical planning and procedural success. Several innovative technological tools have been proposed, including three-dimensional virtual models (3DVMs).¹ Their routine use could enhance preoperative and intraoperative guidance, broadening the indication for RAPN.²

The primary aim of the present study was to evaluate the inter-rater agreement of urologists interpreting conventional CT scans versus hyperaccuracy (HA)-3DVMs (MEDICS Srl, Turin, Italy) to guide preoperative planning for renal masses.

A prospectively maintained database of patients undergoing kidney surgery for renal masses at our Institution was queried to retrieve data of all consecutive RAPN interventions, with preoperative CT scans and HA-3DVMs available. Patients with a history of prior renal surgery and bilateral or multiple ipsilateral tumours were discarded. Performance of CT scans followed a standard internal protocol for staging solid renal masses, using a contrast-medium multidetector CT with 3/5-mm sections from the pulmonary base to the pelvis, with basal, arterial, venous, and excretory phases. The respective HA-3DVMs were developed in selected cases as previously described,³ based on the expected surgical complexity.

The primary outcome was interobserver agreement across 12 specific preoperative surgical planning domains, assessed using a custom-designed questionnaire (Supplementary Material).⁴ The questionnaire comprised 12 items covering different aspects of preoperative surgical planning. Twelve urologists, including six residents and six experienced practitioners, evaluated each clinical case using both the CT scan and the respective HA-3DVMs.

Interobserver agreement was measured using Cohen's kappa (k) statistics for each domain across multiple raters, with 95% confidence intervals (CI) determined by 1000 bootstrap repetitions. Kappa values could range from 0 to 1, with agreement defined as almost perfect (k > 0.8), substantial (k > 0.6), moderate (k > 0.4), fair (k > 0.2), or none to slight (k < 0.2).⁵ Each clinician evaluated all 31 cases twice: once using CT scans and once with HA-3DVMs. The order of evaluation (CT first or HA-3DVM first) was left to clinicians' preference. Each imaging modality was reviewed independently, without back-to-back comparisons, to ensure an unbiased assessment. The time required to complete the evaluation of either the CT scan or HA-3DVMs was recorded. Differences in evaluation times were estimated using the Wilcoxon signed-rank test. A subgroup analysis was conducted to evaluate the interobserver agreement separately for attendings and residents, stratified by imaging modality, to identify potential differences in agreement patterns between these subgroups. The Stata® 17.0 software (StataCorp LLC, College Station, TX, USA) was used for statistical analysis with statistical significance set at p < 0.05.

A HA-3DVM was performed for 31 patients. The median tumour size was 6 cm (IQR 3.5–7.2). RAPN was successfully performed in 25 patients (80.6%). In 6 patients, radical nephrectomy was performed instead. Cohen's kappa coefficient showed higher agreement for HA-3DVMs across most questions, except for those concerning the involvement of the collecting system (Q7, kappa coefficient: 0.37 vs. 0.36) and potential intraoperative opening of the excretory structures (Q12, kappa coefficient: 0.30 vs. 0.26), where the coefficients were nearly identical. Interobserver agreement based on preoperative CT scans was moderate for items concerning tumour position (Q1–3), exophytic/endophytic properties (Q4) and clamping strategy (Q10), and fair or lower for the remaining questions. In contrast, interobserver agreement for HA-3DVMs was substantial for the tumour's longitudinal position (Q1) and its exophytic/endophytic properties (Q4). Moderate agreement was observed for the antero-posterior (Q2) and lateral (Q3) positions, as well as for the necessity and duration of vascular clamping (Q10), while a fair agreement was found for the rest of the questions (Q5–9, Q11–14) (Figure 1). When comparing attendings and residents, interobserver agreement was substantial for HA-3DVM interpretation (attendings: κ = 0.69; residents: κ = 0.67), irrespective of physician status, compared to a moderate to fair agreement for CT scan evaluation (attendings: κ = 0.41; residents: κ = 0.29). Furthermore, nearly all examiners (10/12, 83.3%) required significantly less time to review HA-3DVMs than CT scan imaging.

According to our findings, HA-3DVMs showed higher interobserver agreement than CT scans and shorter evaluation times. To our knowledge, this study represents one of the few attempts to compare the interobserver agreement between CT scans and HA-3DVMs. These results provide additional evidence supporting the role of HA-3DVMs as a facilitator of surgical planning.

This virtual technology facilitates the understanding of anatomical details by eliminating the mental translation of two-dimensional cross-sectional imaging into three-dimensional spatial structures of the kidney. Consequently, it overcomes the loss of spatial depth perception associated with two-dimensional imaging, enabling direct interaction between the surgeon and kidney in three dimensions.⁶ In the present study, the highest interobserver agreement was recorded for tumour topography and clamping strategy, attributable to reduced subjectivity in the imaging review. Moreover, when comparing interobserver agreement by exam type between attendings and residents, it was higher for HA-3DVMs, irrespective of physician status. This suggests that virtual models may enhance interobserver agreement, regardless of evaluators' experience. Our findings align with existing literature, according to which the primary benefits related to presurgical 3DVMs use lie in the in-depth understanding of the renal mass position, with an improvement in the quality of the resection and an increased adoption of a selective clamping strategy.^{1, 2} Through a realistic perception of three-dimensionality, virtual models increase engagement between the examiner and the displayed image. Moreover, the possibility of breaking the model down into its essential components enables precise reconstruction of the spatial relationships between renal mass and anatomy.

Another point worth discussing is the evaluation time, which was significantly shorter for HA-3DVMs for more than 80% of the examiners. These differences highlight that HA-3DVMs offer a more efficient understanding of the anatomy “at a glance” than interpreting multiple plain images. Indeed, according to our results, trainees had evaluation times that closely resembled those of more experienced urologists, suggesting improved navigation of the images, despite lesser surgical experience. This enhanced efficiency can improve confidence in the surgical approach among less experienced surgeons, with great utility in the training setting.⁷ Given the substantial learning curve related to RAPN,⁸ implementing HA-3DVMs can make teaching efforts more effective, expedite learning, and overcome challenges related to robotic training, such as competing resources and increased costs.^{9, 10}

Furthermore, the technical complexity of RAPN for endophytic masses can explain the increased renal damage compared to ablative approaches.¹¹ Several intraoperative factors, such as vascular clamping¹² and the tension created by renorrhaphy¹³ on healthy parenchyma, are under the spotlight for their contribution to additional tissue loss. Consequently, ongoing interest is in modifying this technique to minimize damage to healthy tissue. In this context, adding HA-3DVMs to the preoperative work-up can represent a valuable tool in the urologist's armamentarium, maximizing procedural efficiency and enhancing surgical and functional outcomes.

In conclusion, our study highlights the advantages of incorporating HA-3DVMs into the preoperative assessment of renal masses. Enhanced interobserver agreement and reduced evaluation times underscore the potential of HA-3DVMs to offer a more intuitive and accurate surgical approach, improving preoperative and intraoperative decision-making.

Conception and design: Francesco Ditonno, Alessandro Veccia, Alessandro Antonelli. Acquisition of data: Francesco Ditonno, Michele Boldini, Francesco Cianflone, Lorenzo Treccani, Lorenzo De Bon, Francesca Fumanelli, Francesco Artoni, Claudio Brancelli, Iolanda Palumbo, Alberto Baielli, Alberto Bianchi, Filippo Migliorini, Riccardo Bertolo, Alessandro Veccia, Alessandro Antonelli. Analysis and interpretation of data: Francesco Ditonno, Alessandro Veccia. Drafting of the manuscript: Francesco Ditonno. Critical revision of the manuscript for important intellectual content: Francesco Ditonno, Michele Boldini, Francesco Cianflone, Lorenzo Treccani, Lorenzo De Bon, Francesca Fumanelli, Francesco Artoni, Claudio Brancelli, Iolanda Palumbo, Alberto Baielli, Alberto Bianchi, Filippo Migliorini, Riccardo Bertolo, Alessandro Veccia, Alessandro Antonelli. Supervision: Alessandro Antonelli. Final approval of the version to be published: Francesco Ditonno, Michele Boldini, Francesco Cianflone, Lorenzo Treccani, Lorenzo De Bon, Francesca Fumanelli, Francesco Artoni, Claudio Brancelli, Iolanda Palumbo, Alberto Baielli, Alberto Bianchi, Filippo Migliorini, Riccardo Bertolo, Alessandro Veccia, Alessandro Antonelli. Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved: Francesco Ditonno, Michele Boldini, Francesco Cianflone, Lorenzo Treccani, Lorenzo De Bon, Francesca Fumanelli, Francesco Artoni, Claudio Brancelli, Iolanda Palumbo, Alberto Baielli, Alberto Bianchi, Filippo Migliorini, Riccardo Bertolo, Alessandro Veccia, Alessandro Antonelli.

The authors have no conflict of interest to declare.

All patients provided informed consent before inclusion in the study. The study was conducted in accordance with the principles of the Helsinki Declaration.