Classify with caution: An illustrative example using mixture models and machine learning

The present study compared latent mixture modeling to machine learning classification algorithms using simulated data to evaluate the strengths and weaknesses of alternative classification options for classifying individuals into a relatively low incidence (10%) personality profile. The population model specified that item responses were generated from five latent factors patterned after the Big Five. The simulation varied the number of indicators per factor, factor mean difference, factor variances, and residual item variances to evaluate ten classification techniques, including traditional and Bayesian latent class analysis (LCA), 2 class 90/10 proportion, and factor mixture models, classification trees, conditional inference trees, evolutionary trees, Ward’s hierarchical clustering, K-means, and K-medians techniques. Although classification trees generally outperformed the other techniques, none of the ten techniques resulted in high enough classification accuracy for diagnostic decision making. Classification methods with explanatory and predictive utility may not exhibit adequate diagnostic accuracy for individual decision making.