{"title":"Understanding what industry wants from requirements engineers: an exploration of RE jobs in Canada","authors":"Chong Wang, Peng Cui, M. Daneva, M. Kassab","doi":"10.1145/3239235.3268916","DOIUrl":"https://doi.org/10.1145/3239235.3268916","url":null,"abstract":"[Background] Prior research on the professional occupation of Requirements Engineering (RE) in Europe and Latin America indicated incongruities between RE practice as perceived by industry and as in textbooks, and conducted detailed analysis of both RE and non-RE job aspects. Relatively little is published on the RE competencies and skills industry expects, and seldom investigated the application domains calling for RE professionals. [Aims] We felt motivated by those findings to carry out research on RE job posts in a North-American market. Especially, we focused solely on RE-specific tasks, competencies and skills, from the perspective of defined position categories. Plus, we intend to explore the application domains in need for RE professionals to reveal the wide range of RE roles in industry. [Methods] Coding process, analysis, and synthesis were applied to the textual descriptions of the 190 RE job ads from Canada's most popular online job search site, especially to the text referring to tasks and competencies. [Results] We contribute to the empirical analysis of RE jobs, by providing insights from Canada's IT market in 2017. Using 109 RE job ads from the most popular IT job search portal T-Net, we identified the qualifications, experience and skills demanded by Canadian employers. Furthermore, we explored the distribution of those RE tasks and competences over the 11 categories of RE roles. [Conclusions] Our results suggest that the majority of the employers were big to very big companies in 29 business domains, and the most in-demand RE skills for them were related to RE methods and to project management aspects affecting requirements. In addition, employers placed much more emphasis on experience - both RE-specific and broad software engineering experience, than on higher education.","PeriodicalId":303638,"journal":{"name":"Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"45 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124034818","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
A. V. D. Heijden, Cosmin Broasca, Alexander Serebrenik
{"title":"An empirical perspective on security challenges in large-scale agile software development","authors":"A. V. D. Heijden, Cosmin Broasca, Alexander Serebrenik","doi":"10.1145/3239235.3267426","DOIUrl":"https://doi.org/10.1145/3239235.3267426","url":null,"abstract":"Background Agile methods have been shown to have a negative impact on security. Several studies have investigated challenges in aligning security practices with agile methods, however, none of these have examined security challenges in the context of large-scale agile. Large-scale agile can present unique challenges, as large organizations often involve highly interdependent teams that need to align with other (non-agile) departments. Goal Our objective is to identify security challenges encountered in large-scale agile software development from the perspective of agile practitioners. Method Cooperative Method Development is applied to guide a qualitative case study at Rabobank, a Dutch multinational banking organization. A total of ten interviews is conducted with members in different agile roles from five different agile development teams. Data saturation has been obtained. By open card sorting we identify challenges pertaining to security in agile. Results The following challenges appear to be unique to large-scale agile: alignment of security objectives in a distributed setting, developing a common understanding of the roles and responsibilities in security activities, and integration of low-overhead security testing tools. Additional challenges reported appear to be common to security in software development in general or concur with challenges reported for small-scale agile. Conclusions The reported findings suggest the presence of multiple security challenges unique to large-scale agile. Future work should focus on confirming these challenges and investigating possible mitigations.","PeriodicalId":303638,"journal":{"name":"Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"14 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123614776","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"When and who leaves matters: emerging results from an empirical study of employee turnover","authors":"Panagiota Chatzipetrou, Darja Šmite, R. Solingen","doi":"10.1145/3239235.3267431","DOIUrl":"https://doi.org/10.1145/3239235.3267431","url":null,"abstract":"Background: Employee turnover in GSD is an extremely important issue, especially in Western companies offshoring to emerging nations. Aims: In this case study we investigated an offshore vendor company and in particular whether the employees' retention is related with their experience. Moreover, we studied whether we can identify a threshold associated with the employees' tendency to leave the particular company. Method: We used a case study, applied and presented descriptive statistics, contingency tables, results from Chi-Square test of association and post hoc tests. Results: The emerging results showed that employee retention and company experience are associated. In particular, almost 90% of the employees are leaving the company within the first year, where the percentage within the second year is 50-50%. Thus, there is an indication that the 2 years' time is the retention threshold for the investigated offshore vendor company. Conclusions: The results are preliminary and lead us to the need for building a prediction model which should include more inherent characteristics of the projects to aid the companies avoiding massive turnover waves.","PeriodicalId":303638,"journal":{"name":"Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128909278","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ayushi Rastogi, Nachiappan Nagappan, Georgios Gousios, A. Hoek
{"title":"Relationship between geographical location and evaluation of developer contributions in github","authors":"Ayushi Rastogi, Nachiappan Nagappan, Georgios Gousios, A. Hoek","doi":"10.1145/3239235.3240504","DOIUrl":"https://doi.org/10.1145/3239235.3240504","url":null,"abstract":"Background Open source software projects show gender bias suggesting that other demographic characteristics of developers, like geographical location, can negatively influence evaluation of contributions too. Aim This study contributes to this emerging body of knowledge in software development by presenting a quantitative analysis of the relationship between the geographical location of developers and evaluation of their contributions on GitHub. Method We present an analysis of 70,000+ pull requests selected from 17 most actively participating countries to model the relationship between the geographical location of developers and pull request acceptance decision. Results and Conclusion We observed structural differences in pull request acceptance rates across 17 countries. Countries with no apparent similarities such as Switzerland and Japan had one of the highest pull request acceptance rates while countries like China and Germany had one of the lowest pull request acceptance rates. Notably, higher acceptance rates were observed for all but one country when pull requests were evaluated by developers from the same country.","PeriodicalId":303638,"journal":{"name":"Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129316295","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Bowen Xu, Amirreza Shirani, D. Lo, Mohammad Amin Alipour
{"title":"Prediction of relatedness in stack overflow: deep learning vs. SVM: a reproducibility study","authors":"Bowen Xu, Amirreza Shirani, D. Lo, Mohammad Amin Alipour","doi":"10.1145/3239235.3240503","DOIUrl":"https://doi.org/10.1145/3239235.3240503","url":null,"abstract":"Background Xu et al. used a deep neural network (DNN) technique to classify the degree of relatedness between two knowledge units (question-answer threads) on Stack Overflow. More recently, extending Xu et al.'s work, Fu and Menzies proposed a simpler classification technique based on a fine-tuned support vector machine (SVM) that achieves similar performance but in a much shorter time. Thus, they suggested that researchers need to compare their sophisticated methods against simpler alternatives. Aim The aim of this work is to replicate the previous studies and further investigate the validity of Fu and Menzies' claim by evaluating the DNN- and SVM-based approaches on a larger dataset. We also compare the effectiveness of these two approaches against SimBow, a lightweight SVM-based method that was previously used for general community question-answering. Method We (1) collect a large dataset containing knowledge units from Stack Overflow, (2) show the value of the new dataset addressing shortcomings of the original one, (3) re-evaluate both the DNN-and SVM-based approaches on the new dataset, and (4) compare the performance of the two approaches against that of SimBow. Results We find that: (1) there are several limitations in the original dataset used in the previous studies, (2) effectiveness of both Xu et al.'s and Fu and Menzies' approaches (as measured using F1-score) drop sharply on the new dataset, (3) similar to the previous finding, performance of SVM-based approaches (Fu and Menzies' approach and SimBow) are slightly better than the DNN-based approach, (4) contrary to the previous findings, Fu and Menzies' approach runs much slower than DNN-based approach on the larger dataset - its runtime grows sharply with increase in dataset size, and (5) SimBow outperforms both Xu et al. and Fu and Menzies' approaches in terms of runtime. Conclusion We conclude that, for this task, simpler approaches based on SVM performs adequately well. We also illustrate the challenges brought by the increased size of the dataset and show the benefit of a lightweight SVM-based approach for this task.","PeriodicalId":303638,"journal":{"name":"Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127037403","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pooyan Behnamghader, Patavee Meemeng, Iordanis Fostiropoulos, Di Huang, Kamonphop Srisopha, B. Boehm
{"title":"A scalable and efficient approach for compiling and analyzing commit history","authors":"Pooyan Behnamghader, Patavee Meemeng, Iordanis Fostiropoulos, Di Huang, Kamonphop Srisopha, B. Boehm","doi":"10.1145/3239235.3239237","DOIUrl":"https://doi.org/10.1145/3239235.3239237","url":null,"abstract":"Background: Researchers oftentimes measure quality metrics only in the changed files when analyzing software evolution over commit-history. This approach is not suitable for compilation and using program analysis techniques that require byte-code. At the same time, compiling the whole software not only is costly but may also leave us with many uncompilable and unanalyzed revisions. Aims: We intend to demonstrate if analyzing changes in a module results in achieving a high compilation ratio and a better understanding of software quality evolution. Method: We conduct a large-scale multi-perspective empirical study on 37838 distinct revisions of the core module of 68 systems across Apache, Google, and Netflix to assess their compilability and identify when the software is uncompilable as a result of a developer's fault. We study the characteristics of uncompilable revisions and analyze compilable ones to understand the impact of developers on software quality. Results: We achieve high compilation ratios: 98.4% for Apache, 99.0% for Google, and 94.3% for Netflix. We identify 303 sequences of uncompile commits and create a model to predict uncompilability based on commit metadata with an F1-score of 0.89 and an AUC of 0.96. We identify statistical differences between the impact of affiliated and external developers of organizations. Conclusions: Focusing on a module results in a more complete and accurate software evolution analysis, reduces the cost and complexity, and facilitates manual inspection.","PeriodicalId":303638,"journal":{"name":"Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128152645","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Decision making and visualizations based on test results","authors":"P. Strandberg, W. Afzal, Daniel Sundmark","doi":"10.1145/3239235.3268921","DOIUrl":"https://doi.org/10.1145/3239235.3268921","url":null,"abstract":"Background: Testing is one of the main methods for quality assurance in the development of embedded software, as well as in software engineering in general. Consequently, test results (and how they are reported and visualized) may substantially influence business decisions in software-intensive organizations. Aims: This case study examines the role of test results from automated nightly software testing and the visualizations for decision making they enable at an embedded systems company in Sweden. In particular, we want to identify the use of the visualizations for supporting decisions from three aspects: in daily work, at feature branch merge, and at release time. Method: We conducted an embedded case study with multiple units of analysis by conducting interviews, questionnaires, using archival data and participant observations. Results: Several visualizations and reports built on top of the test results database are utilized in supporting daily work, merging a feature branch to the master and at release time. Some important visualizations are: lists of failing test cases, easy access to log files, and heatmap trend plots. The industrial practitioners perceived the visualizations and reporting as valuable, however they also mentioned several areas of improvement such as better ways of visualizing test coverage in a functional area as well as better navigation between different views. Conclusions: We conclude that visualizations of test results are a vital decision making tool for a variety of roles and tasks in embedded software development, however the visualizations need to be continuously improved to keep their value for its stakeholders.","PeriodicalId":303638,"journal":{"name":"Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"6 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"115478211","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Is there a \"golden\" feature set for static warning identification?: an experimental evaluation","authors":"Junjie Wang, Song Wang, Qing Wang","doi":"10.1145/3239235.3239523","DOIUrl":"https://doi.org/10.1145/3239235.3239523","url":null,"abstract":"Background: The most important challenge regarding the use of static analysis tools (e.g., FindBugs) is that there are a large number of warnings that are not acted on by developers. Many features have been proposed to build classification models for the automatic identification of actionable warnings. Through analyzing these features and related studies, we observe several limitations that make the users lack practical guides to apply these features. Aims: This work aims at conducting a systematic experimental evaluation of all the public available features, and exploring whether there is a golden feature set for actionable warning identification. Method: We first conduct a systematic literature review to collect all public available features for warning identification. We employ 12 projects with totally 60 revisions as our subject projects. We then implement a tool to extract the values of all features for each project revision to prepare the experimental data. Results: Experimental evaluation on 116 collected features demonstrates that there is a common set of features (23 features) which take effect in warning identification for most project revisions. These features can achieve satisfied performance with far less time cost for warning identification. Conclusions: These commonly-selected features can be treated as the golden feature set for identifying actionable warnings. This finding can serve as a practical guideline for facilitating real-world warning identification.","PeriodicalId":303638,"journal":{"name":"Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128162846","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Are 20% of files responsible for 80% of defects?","authors":"Neil Walkinshaw, Leandro L. Minku","doi":"10.1145/3239235.3239244","DOIUrl":"https://doi.org/10.1145/3239235.3239244","url":null,"abstract":"Background: Over the past two decades a mixture of anecdote from the industry and empirical studies from academia have suggested that the 80:20 rule (otherwise known as the Pareto Principle) applies to the relationship between source code files and the number of defects in the system: a small minority of files (roughly 20%) are responsible for a majority of defects (roughly 80%). Aims: This paper aims to establish how widespread the phenomenon is by analysing 100 systems (previous studies have focussed on between one and three systems), with the goal of whether and under what circumstances this relationship does hold, and whether the key files can be readily identified from basic metrics. Method: We devised a search criterion to identify defect fixes from commit messages and used this to analyse 100 active Github repositories, spanning a variety of languages and domains. We then studied the relationship between files, basic metrics (churn and LOC), and defect fixes. Results: We found that the Pareto principle does hold, but only if defects that incur fixes to multiple files count as multiple defects. When we investigated multi-file fixes, we found that key files (belonging to the top 20%) are commonly fixed alongside other much less frequently-fixed files. We found LOC to be poorly correlated with defect proneness, Code Churn was a more reliable indicator, but only for extremely high values of Churn. Conclusions: It is difficult to reliably identify the \"most fixed\" 20% of files from basic metrics. However, even if they could be reliably predicted, focussing on them would probably be misguided. Although fixes will naturally involve files that are often involved in other fixes too, they also tend to include other less frequently-fixed files.","PeriodicalId":303638,"journal":{"name":"Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128503213","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
H Zhao, Jimin Liang, Xuezhen Yin, Lingfeng Yang, Peili Yang, Yuhang Wang
{"title":"Domain-specific modelware: to make the machine learning model reusable and reproducible","authors":"H Zhao, Jimin Liang, Xuezhen Yin, Lingfeng Yang, Peili Yang, Yuhang Wang","doi":"10.1145/3239235.3267439","DOIUrl":"https://doi.org/10.1145/3239235.3267439","url":null,"abstract":"Machine learning task is a routine process including data collection, feature engineering, model training, hyper-parameters tuning, model evaluation and model deployment. The process is usually complex, iterated and time-consuming. Commonly, researchers seldom start building the machine model from scratch. They may select some well-known and well-trained models in similar task domains as the reference models. Then they try to tune the hyper-parameters and accelerate the iteration. Thus, some models are often reused and need to be reproduced by using new training dataset. Moreover, understanding the model and the iteration is more necessary. This scenario is very similar to that of software reuse. In this poster, we propose Modelware and argue the need of Modelware to make the machine learning model reusable and reproducible. We define the Modelware which is the reused object and develop a model repository to provide the model lineage management and model visit tool. The big data for building model is managed collaboratively so that the model can be reproduced. The iteration process to obtain the final optimized model is abstracted and implemented using a lightweight workflow. Finally, we take two different classification tasks as the demonstration.","PeriodicalId":303638,"journal":{"name":"Proceedings of the 12th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement","volume":"61 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2018-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126369615","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}