Comparative Analysis of the Accuracy of Microsoft Excel Macros in Retrospective Chart Review Studies

Introduction

While retrospective chart review is a useful methodology for clinical research, challenges still exist when abstracting data from the electronic health record. When collected manually, unstructured “free text” data are particularly tedious and can be susceptible to errors and biases. We aimed to evaluate the accuracy of Microsoft Excel macros to facilitate the data abstraction process.

Methods

One hundred pathology reports following surgery for thyroid cancer were retrospectively evaluated. Twenty variables of interest (tumor characteristics, invasive features, and lymph node counts) were manually abstracted by a physician reviewer. A macro (“ThyMAC”) was developed to extract the same variables. Abstraction error rates and speed were measured between manual and macro-assisted methods using a paired t-test. Accuracy, classification rates, and interrater reliability of ThyMAC were then analyzed. After identifying correctable errors, an ad hoc analysis of the optimized macro was then performed.

Results

Abstraction errors by physician reviewer were slightly lower relative to ThyMAC (3.8 versus 5.3% error rate, P = 0.03). By contrast, data collection time was 270 times faster via macro-assistance (65 versus 0.24 s per pathology report, P < 0.001). Overall, ThyMAC performed with high rates of accuracy (87-100%) for all abstracted variables, with moderate-to-perfect agreement for 14 of 20 variables. Addressing correctable errors significantly decreased macro error rates compared to the physician abstractor (3.6 versus 0.5%, P < 0.001).

Conclusions

Compared to a trained physician abstractor, macros can extract unstructured data in retrospective chart review studies with high accuracy at speeds superior to a manual approach. Macro errors are typically preventable, and the program can be modified to improve data extraction accuracy. Macros can serve as an efficient and versatile tool to assist researchers with chart review data collection, especially when large datasets are involved.