Detection of outliers with respect to a MUSIC geotechnical database

Abstract

This paper proposes a novel method that addresses a non-traditional class of outlier detection problems. The purpose of most outlier detection methods in the literature is to detect outliers within a dataset. A record can be considered as an outlier if it is distinct from the regular records in the dataset. However, the purpose of the novel outlier detection method proposed by this paper is to detect outlier data groups (a data group may denote a site or a project) with respect to a soil/rock property database. A data group is an outlier group if its characteristics (mean, variance, correlation, or higher order dependency) are distinct from the regular data groups in the database. This paper frames the outlier detection problem into a formal hypothesis testing problem with the null hypothesis “the target data group is identically distributed as the regular groups in the database”. With the hierarchical Bayesian model (HBM) previously developed by the first two authors, the p-value for this hypothesis testing problem can be estimated rigorously. Numerical and real examples show that the p-value can effectively detect outlier data groups as well as outlier records with respect to a database.