{"title":"A career spent wading through industry's empirical ooze (invited talk)","authors":"B. Curtis","doi":"10.1145/2593690.2593699","DOIUrl":"https://doi.org/10.1145/2593690.2593699","url":null,"abstract":"Sometimes analyzing industrial software data seems like draining a swamp. This keynote describes challenges faced while wading through 5 layers of empirical ooze over 36+ years in the software industry. The first layer seeped from formal experiments. The second layer spilled from a global corporate repository of project benchmark data. The third layer leaked from verbalizations by designers and design teams. The fourth layer was emitted by case studies of process improvement. The fifth layer dripped from structural quality data on large IT systems. Five lessons for separating the ooze from the data will be recounted.","PeriodicalId":200036,"journal":{"name":"Conducting Empirical Studies in Industry","volume":"66 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"116019014","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Lessons learned on applying design science for bridging the collaboration gap between industry and academia in empirical software engineering","authors":"Pilar Rodríguez, P. Kuvaja, M. Oivo","doi":"10.1145/2593690.2593694","DOIUrl":"https://doi.org/10.1145/2593690.2593694","url":null,"abstract":"Collaboration between industry practitioners and researchers has been reported as a challenge to conduct empirical studies in software engineering. Gaining practitioners interest in research activities is challenging in the software development industry characterized by tough competition and short time-to-market. Often, practitioners do not see the direct value in collaborating with researchers. Meanwhile, researchers are frequently frustrated because they have no access to the real-world data that would enable them to develop scientific knowledge. Design science has become a popular research framework in Information Systems. This paper describes our lessons learned on how design science can help to attract and motivate industry practitioners to take part in empirical research. We present our practical experiences with using design science in the context of empirical software engineering, and how the design science framework helped us to bridge the collaborative gap with our industry partners. In illustrating our experiences, we present two case studies conducted in collaboration with Ericsson and Elektrobit in which the design science framework was followed. Based on our experience, design science is a suitable research framework for empirical software engineering since much of the research in the field is constructive. The similarities between the design science practices and the day-to-day work of practitioners in industry make it easy for them to understand the research approach. Thus, it supports their willingness to participate in research activities.","PeriodicalId":200036,"journal":{"name":"Conducting Empirical Studies in Industry","volume":"22 14_suppl 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128206615","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"An exploratory study on improving automotive function specifications","authors":"Andreas Vogelsang","doi":"10.1145/2593690.2593697","DOIUrl":"https://doi.org/10.1145/2593690.2593697","url":null,"abstract":"In automotive system development, function specifications describe the requirements and basic design of a function. They are central artifacts and serve as inputs for several activities. With increasing complexity of functions in a vehicle, function specifications become harder to comprehend, change and validate. \u0000 For the purpose of increasing our understanding of what the reasons for these impediments are and how they are related in order to derive potential for improvement, we conducted a qualitative (grounded theory) study. In this study, we interviewed nine senior practitioners of an automotive company on how they work with function specifications and which problems they encounter. \u0000 In this paper, we show the results of this study and report on experiences and challenges of conducting a grounded theory study in industry.","PeriodicalId":200036,"journal":{"name":"Conducting Empirical Studies in Industry","volume":"17 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128086960","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Empirical evaluation of software testing techniques in an open source fashion","authors":"Sheikh Umar Farooq, Smk Quadri","doi":"10.1145/2593690.2593693","DOIUrl":"https://doi.org/10.1145/2593690.2593693","url":null,"abstract":"Testing technique selection and evaluation remains a key issue in software testing. Industry practitioners need concrete evidence to select proper testing techniques in STLC. Despite the large number of empirical studies which attempt to study the testing techniques’ applicability conditions and allied factors, we are still without realistic and generalized results as studies lack a formal foundation and are not complete in all respects. Additionally, besides varying significantly in terms of parameters they have taken into consideration, many existing studies show contradictory results. Even though the researchers stress on replication of these studies under a common set of guidelines, however, attempts to aggregate results from such replications still has not been fruitful so far. As such, to bridge the gap between researchers and industry professionals, we propose to carry out evaluation of testing techniques on a large scale under a unified framework in an open-source fashion so that the realistic and generalized results are obtained in a shorter span of time.","PeriodicalId":200036,"journal":{"name":"Conducting Empirical Studies in Industry","volume":"59 13","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132575224","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Exploring web advertising to attract industry professionals for software engineering surveys","authors":"M. Galster, Daniel Tofan","doi":"10.1145/2593690.2593695","DOIUrl":"https://doi.org/10.1145/2593690.2593695","url":null,"abstract":"The validity of questionnaire-based surveys in empirical software engineering studies depends on the number of survey participants and the quality of the data collected from participants. Attracting respondents is a continuous challenge and many studies rely on participants from the personal network of researchers. In this short paper we present several insights from using web advertising to recruit participants for one of our surveys on software architecture decision making. More specifically, in this work we show how we used web advertising in addition to contacting our personal networks, postings on mailing lists, blogs, etc. to reach a larger and broader base of our target population. We discuss the lessons we learned through this experience, and potentials and trade-offs of using web advertising in software engineering surveys.","PeriodicalId":200036,"journal":{"name":"Conducting Empirical Studies in Industry","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121486152","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"On the relationship between quality assurance and productivity in software companies","authors":"C. H. C. Duarte","doi":"10.1145/2593690.2593692","DOIUrl":"https://doi.org/10.1145/2593690.2593692","url":null,"abstract":"Quality assurance methods based on software process im- provement models have been regarded as a main source of variability in software productivity. In this paper, we investi- gate the relationship between labor productivity and quality assurance levels, using a data set containing more than 500 Brazilian software rms. We perform statistical analyses relating labor productivity, as measured through the annual gross revenue per worker ratio, to quality levels, whose ma- turity was examined in appraisals performed from 2006 to 2012 according to two distinct software process improvement models (CMMI and MPS.BR). As a preparatory step to our ndings, we investigate the relationship between these mod- els. We show that CMMI and MPS.BR appraised maturity levels are correlated, but we nd no statistical evidence that the implemented quality assurance methods are related to higher labor productivity or productivity growth.","PeriodicalId":200036,"journal":{"name":"Conducting Empirical Studies in Industry","volume":"143 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127171436","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Is the glass half empty or half full? conducting empirical studies in industry (invited talk)","authors":"I. Ozkaya","doi":"10.1145/2593690.2593698","DOIUrl":"https://doi.org/10.1145/2593690.2593698","url":null,"abstract":"Building effective collaborations, scoping a research problem to produce results that will benefit the industry, and lacking access to data are often listed as barriers to conducting academic studies in industry. Despite these challenges, research that uses empirical approaches and that collaborates with industry is increasing; there are even conferences and journals dedicated to the cause. The software engineering research community aims to produce results that are valuable to practice through these endeavors. Yet, one key goal of this research seems to get lost as we try to sort through the challenges that such collaborations impose upon the work plan: Do the results of these studies really help us innovate and produce research outputs that others can use, build on, and replicate? Or are we becoming victims of the available empirical software engineering methods? In this presentation, I will give examples of how the results of empirical studies have helped shape research problems in which industry can more easily take part and even be engaged partners in the work.","PeriodicalId":200036,"journal":{"name":"Conducting Empirical Studies in Industry","volume":"4 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128818010","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Silverio Martínez-Fernández, Helena Martins Marques
{"title":"Practical experiences in designing and conducting empirical studies in industry-academia collaboration","authors":"Silverio Martínez-Fernández, Helena Martins Marques","doi":"10.1145/2593690.2593696","DOIUrl":"https://doi.org/10.1145/2593690.2593696","url":null,"abstract":"More and more, software engineering researchers are motivated to solve real problems that bring value to industry. An example is the industry-academia collaboration described in this paper among everis, an IT consulting firm, and the GESSI research group at UPC. The goal of this paper is twofold: to evaluate the success of the collaboration, and to report the experience with conducting empirical studies in the industry and lessons learnt. We evaluated our collaboration with an existing model for technology transfer, and performed a focus group discussion to identify challenges we have faced. After initialization and alignment of the collaboration, a high maturity level has been achieved: we have obtained the first results in form of proposed solutions, scientific publications, and pilots run in real projects. In spite of this positive progress, further initiatives need to be undertaken in the last phases of the collaboration to achieve high degrees of maturity in deployment impact, industry benefit and innovativeness. Evaluating the collaboration has been positive, since we identified the next steps to be taken to achieve a high degree of technology transfer and innovation dissemination. We think it is a needed step in industry-academia collaborations in order to improve their success.","PeriodicalId":200036,"journal":{"name":"Conducting Empirical Studies in Industry","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129172926","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Topic selection in industry experiments","authors":"Ayse Tosun Misirli, H. Erdogmus, Natalia Juristo Juzgado, Óscar Dieste Tubío","doi":"10.1145/2593690.2593691","DOIUrl":"https://doi.org/10.1145/2593690.2593691","url":null,"abstract":"This paper shares our experience with initial negotiation and topic elicitation process for conducting industry experiments in six software development organizations in Finland. The process involved interaction with company representatives in the form of both multiple group discussions and separate face-to-face meetings. Fitness criteria developed by researchers were applied to the list of generated topics to decide on a common topic. The challenges we faced include diversity of proposed topics, communication gaps, skepticism about research methods, initial disconnect between research and industry needs, and lack of prior work relationship. Lessons learned include having enough time to establish trust with partners, importance of leveraging the benefits of training and skill development that are inherent in the experimental approach, uniquely positioning the experimental approach within the landscape of other validation approaches more familiar to industrial partners, and introducing the fitness criteria early in the process.","PeriodicalId":200036,"journal":{"name":"Conducting Empirical Studies in Industry","volume":"352 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126685968","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Parametric model CESI experience: (keynote)","authors":"B. Boehm","doi":"10.1109/CESI.2013.6618461","DOIUrl":"https://doi.org/10.1109/CESI.2013.6618461","url":null,"abstract":"This presentation will summarize four decades of conducting empirical studies in industry in the area of parametric estimation models. It will identify several critical success factors, and will compare them to those of other types of empirical studies. Finally, synergies about different forms of empirical studies will be discussed.","PeriodicalId":200036,"journal":{"name":"Conducting Empirical Studies in Industry","volume":"2014 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2013-05-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122168154","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}