{"title":"Using templates to elicit implied security requirements from functional requirements - a controlled experiment","authors":"M. Riaz, John Slankas, J. King, L. Williams","doi":"10.1145/2652524.2652532","DOIUrl":"https://doi.org/10.1145/2652524.2652532","url":null,"abstract":"Context: Security requirements for software systems can be challenging to identify and are often overlooked during the requirements engineering process. Existing functional requirements of a system can imply the need for security requirements. Systems having similar security objectives (e.g., confidentiality) often also share security requirements that can be captured in the form of reusable templates and instantiated in the context of a system to specify security requirements.\u0000 Goal: We seek to improve the security requirements elicitation process by automatically suggesting appropriate security requirement templates implied by existing functional requirements.\u0000 Method: We conducted a controlled experiment involving 50 graduate students enrolled in a software security course to evaluate the use of automatically-suggested templates in eliciting implied security requirements. Participants were divided into treatment (automatically-suggested templates) and control groups (no templates provided).\u0000 Results: Participants using our templates identified 42% of all the implied security requirements in the oracle as compared to the control group, which identified only 16% of the implied security requirements. Template usage increased the efficiency of security requirements identified per unit of time.\u0000 Conclusion: Automatically-suggested templates helped participants (security non-experts) think about security implications for the software system and consider more security requirements than they would have otherwise. We found that participants need more incentive than just a participatory grade when completing the task. Further, we recommend to ensure task completeness, participants either need a step-driven (i.e., wizard) approach or progress indicators to identify remaining work.","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"3 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134417127","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dietmar Pfahl, Huishi Yin, M. Mäntylä, Jürgen Münch
{"title":"How is exploratory testing used? A state-of-the-practice survey","authors":"Dietmar Pfahl, Huishi Yin, M. Mäntylä, Jürgen Münch","doi":"10.1145/2652524.2652531","DOIUrl":"https://doi.org/10.1145/2652524.2652531","url":null,"abstract":"Context: Exploratory Testing has experienced a rise in popularity in the industry with the emergence of agile development practices, yet it remains unclear, in which domains and how it is used in practice.\u0000 Goal: To study how software engineers understand and apply the principles of exploratory testing, as well as the specific advantages and difficulties they experience.\u0000 Method: We conducted an online survey in the period June to August 2013 among Estonian and Finnish software developers and testers.\u0000 Results: Our main findings are that the majority of testers, developers, and test managers using ET, (1) apply ET to usability-critical, performance-critical, security-critical and safety-critical software to a high degree; (2) use ET very flexibly in all types of test levels, activities, and phases; (3) perceive ET as an approach that supports creativity during testing and that is effective and efficient; and (4) find that ET is not easy to use and has little tool support.\u0000 Conclusions: The high degree of application of ET in critical domains is particularly interesting and indicates a need for future research to obtain a better understanding of the effects of ET in these domains. In addition, our findings suggest that more support to ET users should be given (guidance and tools).","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"28 1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134318965","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Matteo Giacalone, F. Paci, R. Mammoliti, Rodolfo Perugino, F. Massacci, Claudio Selli
{"title":"Security triage: an industrial case study on the effectiveness of a lean methodology to identify security requirements","authors":"Matteo Giacalone, F. Paci, R. Mammoliti, Rodolfo Perugino, F. Massacci, Claudio Selli","doi":"10.1145/2652524.2652585","DOIUrl":"https://doi.org/10.1145/2652524.2652585","url":null,"abstract":"Context: Poste Italiane is a large corporation offering integrated services in banking and savings, postal services, and mobile communication. Every year, it receives thousands of change requests for its ICT services. Applying to each and every request a security assessment \"by the book\" is simply not possible. Goal: We report the experience by Poste Italiane of a lean methodology to identify security requirements that can be inserted in the production cycle of a normal company. Method: The process is based on surveying the overall IT architectures (Security Survey) and then a lean dynamic process (Security Triage) to evaluate individual change requests, so that important changes get the attention they need, minor changes can be quickly implemented, and compliance and security obligations are met. Results: The empirical evaluation conducted for over an year at Poste Italiane shows that the process significantly reduces the time to identify security requirements at the pace of change. Conclusions: The Security Survey and Triage process should thus be embedded in a company's production cycle as mandatory step to manage change requests so that security initiatives are prioritized based on the relevance of the assets and of the business objectives of the company.","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"78 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128036264","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
D. Ribeiro, Marcos Cardoso, F. Silva, A. C. A. França
{"title":"Using qualitative metasummary to synthesize empirical findings in literature reviews","authors":"D. Ribeiro, Marcos Cardoso, F. Silva, A. C. A. França","doi":"10.1145/2652524.2652562","DOIUrl":"https://doi.org/10.1145/2652524.2652562","url":null,"abstract":"Context- A common problem in Systematic Reviews in software engineering is that they provide very limited syntheses. Goal- In the search for alternatives of effective methods for synthesizing empirical evidence, in this paper, we explore the use of the Qualitative Metasummary method, which is a quantitatively oriented aggregation of mixed research findings. Method - We describe the use of qualitative metasummary through an example using 15 studies addressing antecedents of performance of software development teams. Qualitative metasummary includes extraction and grouping of findings, and calculation of frequency and intensity effect sizes. Results -- The instance described in this paper produced a 10-factor model that effectively summarizes the current empirical knowledge on performance of software development teams. Then, we assessed the method in terms of ease of use, usefulness and reliability of results. Conclusion -- The Qualitative Metasummary method offers rich indexes of experiences and events under investigation, focusing on the effects of a variable over other, which is consistent with the central interest of systematic reviews. However, its main limitations are (i) challenging comparability/integratability between primary studies, (ii) loss of detailed contextual information, (iii) and the great deal of effort demanded to synthesize larger sets of papers.","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126882104","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Estimation of student's programming expertise","authors":"Eduard Kuric, M. Bieliková","doi":"10.1145/2652524.2652561","DOIUrl":"https://doi.org/10.1145/2652524.2652561","url":null,"abstract":"Context: Despite the fact, that the various automated expertise metrics were proposed, we do not know which metrics the most reliably capture/reflect expertise. Goal: To define metrics for estimation of developer's expertise based on programming tasks, to evaluate which of them most reliably capture expertise, and to propose and evaluate an automatic process to compare the metrics. Method: We define three expertise metrics with respects to such characteristics as spent time, performed activities and complexity of source code. We evaluate Spearman's correlation between our expertise metrics and students' score obtained after completion of a programming course with 251 students. Results: The best (very strong) correlation is between the metrics based on complexity of source code and the student's qualification points. Conclusions: Very strong but not perfect correlation is between our estimation of student's expertise and his/her score in the second third of the course. Approximately in the middle of the course we might be able to predict students' grades.","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"129690273","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Factors impacting rapid releases: an industrial case study","authors":"Noureddine Kerzazi, Foutse Khomh","doi":"10.1145/2652524.2652589","DOIUrl":"https://doi.org/10.1145/2652524.2652589","url":null,"abstract":"Context: Software release teams try to reduce the time needed for the transit of features or bug fixes from the development environment to the production, crossing all the quality gates. However, little is known about the factors that influence the time-to-production and how they might be controlled in order to speed up the release cycles.\u0000 Goal: This paper examines step by step the release process of an industrial software organization aiming to identify factors that have a significant impact on the lead time and outcomes of the software releases.\u0000 Method: Over 14 months of release data have been analyzed (246 releases from the isolated source code branches to the production environment).\u0000 Results: We discuss three dimensions under which a series of factors could be addressed: technical, organizational, and interactional. We present our findings in terms of implications for release process improvements.\u0000 Conclusions: Our analyzes reveal that testing is the most time consuming activities (86%) along with the need for more congruence among teams, especially in the context of parallel development.","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121732586","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"How do code refactorings affect energy usage?","authors":"C. Sahin, L. Pollock, J. Clause","doi":"10.1145/2652524.2652538","DOIUrl":"https://doi.org/10.1145/2652524.2652538","url":null,"abstract":"Context: Code refactoring's benefits to understandability, maintainability and extensibility are well known enough that automated support for refactoring is now common in IDEs. However, the decision to apply such transformations is currently performed without regard to the impacts of the refactorings on energy consumption. This is primarily due to a lack of information and tools to provide such relevant information to developers. Unfortunately, concerns about energy efficiency are rapidly becoming a high priority concern in many environments, including embedded systems, laptops, mobile devices, and data centers.\u0000 Goal: We aim to address the lack of information about the energy efficiency impacts of code refactorings.\u0000 Method: We conducted an empirical study to investigate the energy impacts of 197 applications of 6 commonly-used refactorings.\u0000 Results: We found that refactorings can not only impact energy usage but can also increase and decrease the amount of energy used by an application. In addition, we also show that metrics commonly believed to correlate with energy usage are unlikely to be able to fully predict the impact of applying a refactoring.\u0000 Conclusion: The results from this and similar studies could be used to augment IDEs to help software developers build more energy efficient software.","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128460083","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Fabio Calefato, F. Lanubile, R. Prikladnicki, João Henrique Stocker Pinto
{"title":"An empirical simulation-based study of real-time speech translation for multilingual global project teams","authors":"Fabio Calefato, F. Lanubile, R. Prikladnicki, João Henrique Stocker Pinto","doi":"10.1145/2652524.2652537","DOIUrl":"https://doi.org/10.1145/2652524.2652537","url":null,"abstract":"Context: Real-time speech translation technology is today available but still lacks a complete understanding of how such technology may affect communication in global software projects.\u0000 Goal: To investigate the adoption of combining speech recognition and machine translation in order to overcome language barriers among stakeholders who are remotely negotiating software requirements.\u0000 Method: We performed an empirical simulation-based study including: Google Web Speech API and Google Translate service, two groups of four subjects, speaking Italian and Brazilian Portuguese, and a test set of 60 technical and non-technical utterances.\u0000 Results: Our findings revealed that, overall: (i) a satisfactory accuracy in terms of speech recognition was achieved, although significantly affected by speaker and utterance differences; (ii) adequate translations tend to follow accurate transcripts, meaning that speech recognition is the most critical part for speech translation technology.\u0000 Conclusions: Results provide a positive albeit initial evidence towards the possibility to use speech translation technologies to help globally distributed team members to communicate in their native languages.","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"128980061","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
M. Staples, Ross Jeffery, June Andronick, Toby C. Murray, G. Klein, Rafal Kolanski
{"title":"Productivity for proof engineering","authors":"M. Staples, Ross Jeffery, June Andronick, Toby C. Murray, G. Klein, Rafal Kolanski","doi":"10.1145/2652524.2652551","DOIUrl":"https://doi.org/10.1145/2652524.2652551","url":null,"abstract":"Context: Recent projects such as L4.verified (the verification of the seL4 microkernel) have demonstrated that large-scale formal program verification is now becoming practical.\u0000 Objective: We address an important but unstudied aspect of proof engineering: proof productivity.\u0000 Method: We extracted size and effort data from the history of the development of nine projects associated with L4.verified.\u0000 Results: We find strong linear relationships between effort and proof size for projects and for individuals. We discuss opportunities and limitations with the use of lines of proof as a size measure, and discuss the importance of understanding proof productivity for future research.\u0000 Conclusions: An understanding of proof productivity will assist in its further industrial application and provide a basis for cost estimation and understanding of rework and tool usage.","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"19 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122570485","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Patterns of folder use and project popularity: a case study of github repositories","authors":"Jiaxin Zhu, Minghui Zhou, A. Mockus","doi":"10.1145/2652524.2652564","DOIUrl":"https://doi.org/10.1145/2652524.2652564","url":null,"abstract":"Context: Every software development project uses folders to organize software artifacts. Goal: We would like to understand how folders are used and what ramifications different uses may have. Method: In this paper we study the frequency of folders used by 140k Github projects and use regression analysis to model how folder use is related to project popularity, i.e., the extent of forking. Results: We find that the standard folders, such as document, testing, and examples, are not only among the most frequently used, but their presence in a project is associated with increased chances that a project's code will be forked (i.e., used by others) and an increased number of forks. Conclusions: This preliminary study of folder use suggests opportunities to quantify (and improve) file organization practices based on folder use patterns of large collections of repositories.","PeriodicalId":124452,"journal":{"name":"International Symposium on Empirical Software Engineering and Measurement","volume":"39 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2014-09-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122045331","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}