A Graphical Evaluation Tool to Utilize ECG Data Without Reference Annotation

Abstract

Without reference annotation, statistical metrics such as sensitivity and positive predictive value (PPV) cannot be calculated. Annotating a large ECG database may not be feasible, hence, the interest in developing an evaluation tool that does not require reference annotation. We developed a tool for evaluating key performance attributes (KPA) including arrhythmia detection, heart rate, ST value, and noise tolerance. The tool has three layers of KPA graphics. The top layer includes interactive distribution graphs of the KPA values for aggregated results for the entire database. From this top layer the user can select an individual record to launch interactive trending graphs that display the KPA values, or their discrepancies, for a time span on that particular record. From this second layer the user can identify any KPA value of interest (e.g., a specific arrhythmia label) to view the underlying ECG waveform. Navigating through these three layers, the user is able to quickly confirm the validity of KPA reported by the algorithm. We modified the noise tolerance of an exercise ECG arrhythmia algorithm. Then used this tool to visually verify the resulting improvement on the Telemetric and Holter ECG Warehouse (THEW) stress database E-OTH-12-0927-015. We confirmed the visual verification of improvement by manually annotating a small subset of records in this database.