Methodological challenges and opportunities in evaluating clinical safety in primary eyecare services.

Abstract

To the Editor: The recent paper by Swystun and Davey is a welcome addition to literature on clinical safety in acute eyecare services in primary care. Their research raises important questions about how clinical safety in such services, and indeed primary eyecare services in general, should be determined. Evaluations of effectiveness of UK primary eyecare services have typically focussed on false positives, for example, false positive referrals to the Hospital Eye Service (HES), with false negatives rarely being considered. Indeed, there are interesting methodological challenges in examining for false negatives, but also, we suggest, some novel opportunities with the advent of greater digital interconnectedness in eyecare pathways. In terms of challenges, one difficulty is defining what constitutes a false negative, with different methodologies and their consequent different criterion options for what comprises a false negative potentially being a major source of difference when investigating similar services. In the absence of an established definition of what constitutes a COVID19 Urgent Eyecare Service (CUES) optometrist's incorrect diagnosis or an unsuccessful recommendation, Swystun and Davey were obliged to set their own definitions. They used direct patient contact via telephone following access to CUES, quantifying cases of missed pathology and/or failure to appropriately manage patients’ symptoms in a sample of over a thousand episodes. Their “incorrect” diagnoses were determined by patients’ accounts, but they also went on to define “major errors”, a definition which included errors or omissions judged to have the potential to cause harm. Although the precise definitions of CUES misdiagnoses and unsuccessful recommendations might be open for debate, consensus is more likely to be found regarding a potential for harm category. While the authors describe some limitations of reliance on patient reported outcomes, their point that the purpose of acute eyecare relates to resolving patients’ symptoms highlights the advantage such a direct approach can have in the context of urgent eyecare. Examining clinical records is another approach sometimes employed to establish safety. Interestingly, Sheen et al.’s study examining clinical safety in an urgent eyecare scheme used a combination of clinical record review and telephone interview outcomes to establish inappropriate management, reporting a substantially lower false negative rate than Swystun and Davey, albeit in a different service (not reliant on the substantive telemedicine element within CUES during the pandemic). Konstantakopolou et al. also employed record review to examine clinical safety in a minor eye conditions scheme (MECS), examining not only referred but also nonreferred patients’ records to assess for appropriate management; however, while this approach can be used to capture clinical management against guidelines, it is reliant on the veracity of recorded details, with potential for documentation inconsistencies versus actual presentations, overand underreporting of tests/examinations, and/or symptoms evaluated and/ or advice and management provided. In a recent evaluation of false negatives in CUES (and within the same service and pandemic timeline as Swystun and Davey), Williams et al. tracked a large population of over 1000 cases seen in primary care, monitoring the potential for nonreferred cases to attend an HES emergency eye department within 28 days. Williams et al. offered this criterion as a proxy for possible false negatives following CUES assessment, with the large sample size and single withinarea HES provider accommodating urgent eyecare permitting interrogation of safety. Their analysis compared both primary care clinical records for nonreferred cases and secondary care clinical records for those nonreferred primary care cases subsequently representing to the emergency eye department. Although this method was advantageous in utilising routine data sources within the parameters of service evaluation and audit, it would not have captured nonreferred CUES cases presenting to other providers. In contrast, the method of Swystun and Davey did partially illuminate the outcome of cases directed to care by their General Medical Practitioner and NHS 111, including for example cases of stroke, which may not otherwise have attended ophthalmology. Notably, however, the difference in findings between Williams et al. and Swystun and Davey for CUES is very striking, but potentially somewhat less so when considering those cases with the potential for harm. Primary care services for usually nonurgent and/or often asymptomatic conditions, for example glaucoma, present different methodological challenges when examining clinical safety. In particular, the absence of acute symptoms and/or the timeline for potential changes in status require Published online: 29 January 2022