Calibration and reliability testing of a novel asynchronous photographic plaque scoring system in young children

Objectives

The Simplified Oral Hygiene Index for Maxillary Incisors (OHI-MIS) is a novel plaque scoring system adapted for young children. This study describes calibration training and testing used to establish the inter- and intra-rater reliability for OHI-MIS measured from clinical photographs.

Methods

Two raters from the Coordinated Oral Health Promotion Chicago (CO-OP) and one from the Behavioral EConomics for Oral health iNnovation (BEECON) randomized controlled trials (RCTs) underwent calibration with gold standard raters, followed by annual re-calibration. Raters from CO-OP also completed inter-rater reliability testing; all three raters completed intra-rater reliability testing rounds. Photographs were obtained from children aged 9–39 months.

Results

All three raters achieved greater than 0.77 Lin's Concordance Correlation (LCC) versus gold standard consensus during calibration. All three raters had LCC ≥0.83 at recalibration 1 year later. CO-OP trial raters scored 604 photos (151 sets of 4 photographs); mostly both raters were somewhat/very confident in their scoring (≥89%), describing the most photos as “clear” (90% and 81%). The CO-OP inter-rater LCC for total OHI-MIS score was 0.86, changing little when low quality or confidence photos were removed. All three raters demonstrated high intra-rater reliability (≥0.83).

Conclusions

The OHI-MIS plaque scoring system on photos had good reliability within and between trials following protocol training and calibration. OHI-MIS provides a novel asynchronous plaque scoring system for use in young children. Non-clinicians in field or clinical settings can obtain photographs, offering new opportunities for research and clinical care.