Towards Systematic Benchmarking of Activity Recognition Algorithms

Abstract

In this paper we propose a benchmarking frame-work for evaluating activity recognition methods. We use an ontology for describing activity recognition datasets on the meta-level and propose a fine-grained annotation scheme for activity recognition datasets. Given a method that implements a defined interface, an evaluation client can be used to automatically run experiments on annotated datasets. Our framework helps to find relevant datasets and makes results reproducible by fixing concrete experimental settings. We show how to use the framework and report about a preliminary evaluation experiment.