Response to Second Letter on Response Letter Regarding “An Artificial Neural Network-Based Model to Predict Chronic Kidney Disease in Aged Cats”

Abstract

We are sorry to have disappointed Dr. Wun in not providing him with the numerical information he requested. Our answer did focus on the rationale of our model, which was intended to predict as accurately as possible which cats would progress to azotemic CKD within 12 months of the initial screening in field conditions. We have expanded upon our previous reply in this letter.

Defining what constitutes early kidney disease is not straightforward, and key opinion leaders from the IRIS group offer a number of possible definitions, no doubt because none of these on their own is perfect. In our study, we took the approach of using a subsequent conventional diagnosis of azotemic CKD (based on our laboratory's reference interval for serum creatinine concentration together with USG < 1.035) within 12 months of a single screening event to indicate that, at the screening event, the cat had subclinical early CKD. In our opinion, this is an unambiguous definition of early CKD, whereas most other ways of defining early CKD, without subsequent follow-up confirming a conventional diagnosis of CKD, carry a degree of uncertainty about whether these cats will continue to lose kidney function over time.

Dr. Wun cites Brans et al. (2021) as evidence that a serum creatinine concentration of 1.76 mg/dL has a 92.9% specificity for borderline low GFR (< 1.7 mL/kg/min). This study, which focused on comparing the ability of serum SDMA and creatinine concentrations to identify cats with “borderline low GFR,” included 17 clinically healthy cats with no mention of how their kidney function progressed over time. The “borderline low GFR” definition was based on a previously published study from the same group, which involved 16 healthy clinically normal cats. Such small numbers would not be sufficient to establish a robust reference interval, and using this data to define a cutoff value of serum creatinine concentration with a given sensitivity and specificity to diagnose CKD early seems flawed to us.

Measurement of GFR in clinical practice is laborious and, despite a number of published studies, what constitutes a normal GFR and the variations that occur in a large population of healthy cats are not well defined. If it were, the IRIS group would have adopted GFR to define the stages of CKD and thus better define the non-azotemic stages. Importantly, the variation in GFR within the same cat when measured multiple times remains to be clearly defined. This variation needs to be understood before serial GFR measurements could be used to document deteriorating kidney function over time in early-stage CKD.

Our study was designed to help veterinarians in primary care practice make better use of health screening data to identify cats with a high likelihood of developing azotemic CKD within 12 months of a single visit. Data from 218 healthy cats screened and followed prospectively were analyzed by an artificial neural network to build models that identified those cats that developed azotemic CKD within 12 months. The model with the highest accuracy was tested and further refined using data from another 3546 cats.

If we just used a serum creatinine concentration range > 1.6 but < 1.9 mg/dL to predict which cats in our validation dataset developed azotemic CKD (serum creatinine concentration persistently > 2 mg/dL within 12 months), only 305 of the 1521 cats would be correctly identified. Furthermore, 438 cats out of the 2025 that did not develop azotemic CKD would be incorrectly predicted by using the same serum creatinine concentration range. Thus, this method, when used on our dataset, gave a specificity of 78.4%, but a sensitivity of only 20.1%. To improve the sensitivity of using only serum creatinine concentration would lower specificity, which is always the trade-off. With our published model setting the thresholds to maximize both sensitivity and specificity, we were able to achieve a sensitivity of 87% while maintaining a specificity of 70%. Early diagnosis of CKD, which does not lead to any treatment that carries a risk of harm if applied to patients without the disease, merits models that maximize sensitivity even though doing so may compromise specificity.

We hope that our explanations in response to Dr. Wun's second letter add to the debate on how clinically relevant early CKD can be more readily recognized.

Yours sincerely,