Describing What Experimental Software Engineering Experts Do When They Design Their Experiments - A Qualitative Study

Background: Although there has been a significant amount of research focused on designing and conducting controlled experiments, few studies report how experienced experimental software engineering researchers actually design and conduct their studies. Aims: This study aimed to offer a practical perspective from their viewpoint regarding controlled experiment planning. Method: We collected data through semi-structured interviews from 11 researchers, and we used qualitative analysis methods from the grounded theory approach to analyze them. Result: Although the complete study presents four research questions, in this paper, we answer the first one. As a result, we present a preliminary result about what these experts actually do when they design experiments. Conclusions: This work contributes to a better understanding of the practical performance of experimental software engineering.