ORDIP: Principle, Practice and Guidelines for Open Research Data in Indoor Positioning

Abstract

The community of indoor positioning research has identified the need for a paradigm shift towards more reproducible and open research dissemination. Despite recent efforts to openly share data and code, accompanying research results with Open Research Data (ORD) is far from being the de facto standard option for publications in the indoor positioning field. The lack of recognized public benchmarks and the rather slow adoption of ORD, set a great volume of astute contributions in the field to remain irreproducible. Performance comparisons may often be made on experiments performed in different settings, hindering their consistency, and eventually slowing down progress and the evolution of knowledge in the field. In this work, we systematically review the landscape of Open Research Data in Indoor Positioning, enlisting, presenting, and analyzing the characteristic features of the relevant available open datasets of the field. As a result of our systematic review, the statistical analysis of the 119 identified open datasets, highlights the tendencies and the missing elements, such as underrepresented technologies (such as Ultra-Wideband) and measurement types (such as Angle of Arrival, Time Difference of Arrival). A result that stands out is the frequency of crucial metadata information that remains undefined, such as the size of the area of collection (50% of the datasets), the ground truth collection protocol (21%), or the environment type (13%). As a fruit of the systematic analysis, we discuss potential shortcomings, and we share lessons learned and observed good practices regarding the provision of a new ORD and the reuse of existing ones. A significant practical contribution of this work is a list of guidelines that researchers aiming to collect and share a new ORD can follow as a simple checklist. In a broader context, we consider that ORDIP can help measure the future progress of the Indoor Positioning field in the ORD front through the snapshot of the current landscape that it provides. The Open provision of our full systematic analysis of the ORD can serve as a look-up table for easy access to the ORDs containing the most relevant features for each interested researcher, while our guidelines aim to support the community and spark the discussion towards a consensus-based standard for ORD of the field.