Dataset Coverage for Testing Machine Learning Computer Programs

Abstract

Machine learning programs are non-testable, and thus testing with pseudo oracles is recommended. Although metamorphic testing is effective for testing with pseudo oracles, identifying metamorphic properties has been mostly ad hoc. This paper proposes a systematic method to derive a set of metamorphic properties for machine learning classifiers, support vector machines. The proposal includes a new notion of test coverage for the machine learning programs; this test coverage provides a clear guideline for conducting a series of metamorphic testing.