ACM Transactions on Computing Education最新文献

{"title":"Combining GitHub, Chat, and Peer Evaluation Data to Assess Individual Contributions to Team Software Development Projects","authors":"C. Hundhausen, Phill Conrad, Olusola O. Adesope, Ahsun Tariq","doi":"10.1145/3593592","DOIUrl":"https://doi.org/10.1145/3593592","url":null,"abstract":"Assessing team software development projects is notoriously difficult and typically based on subjective metrics. To help make assessments more rigorous, we conducted an empirical study to explore relationships between subjective metrics based on peer and instructor assessments, and objective metrics based on GitHub and chat data. We studied 23 undergraduate software teams (n = 117 students) from two undergraduate computing courses at two North American research universities. We collected data on teams’ (a) commits and issues from their GitHub code repositories, (b) chat messages from their Slack and Microsoft Teams channels, (c) peer evaluation ratings from the CATME peer evaluation system, and (d) individual assignment grades from the courses. We derived metrics from (a) and (b) to measure both individual team members’ contributions to the team, and the equality of team members’ contributions. We then performed Pearson analyses to identify correlations among the metrics, peer evaluation ratings, and individual grades. We found significant positive correlations between team members’ GitHub contributions, chat contributions, and peer evaluation ratings. In addition, the equality of teams’ GitHub contributions was positively correlated with teams’ average peer evaluation ratings and negatively correlated with the variance in those ratings. However, no such positive correlations were detected between the equality of teams’ chat contributions and their peer evaluation ratings. Our study extends previous research results by providing evidence that (a) team members’ chat contributions, like their GitHub contributions, are positively correlated with their peer evaluation ratings; (b) team members’ chat contributions are positively correlated with their GitHub contributions; and (c) the equality of team’ GitHub contributions is positively correlated with their peer evaluation ratings. These results lend further support to the idea that combining objective and subjective metrics can make the assessment of team software projects more comprehensive and rigorous.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"23 1","pages":"1 - 23"},"PeriodicalIF":2.4,"publicationDate":"2023-05-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41635789","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Global Manager: A Serious Game to Raise Awareness of the Challenges of Being a Project Manager in Global Software Development","authors":"A. Vizcaíno, F. García, V. Menéndez, Antonio Manjavacas, Rubén Márquez, Marta Molina","doi":"10.1145/3592620","DOIUrl":"https://doi.org/10.1145/3592620","url":null,"abstract":"Project managers tend to confront multiple challenges in Global Software Development (GSD), including misunderstandings about the project requirements, complex estimations of costs, risks, and efforts, along with increases in task allocation and a lack of coordination. “Soft skills” play a fundamental role in solving these challenges, as they are the human or social abilities that facilitate the resolution of conflicts and involve dealing with people and managing projects in their most social and creative dimension. However, developing the soft skills that allow project managers to lead in an effective manner is not an easy task, especially in global or distributed environments, where coordination and control problems are commonplace. One suitable mechanism with which to develop these soft skills is that of serious games, which focus on learning by doing and allow users to simulate real scenarios. The authors of this article are aware of the usefulness of this type of teaching method and have, therefore, developed the serious game described herein. This serious game is denominated as Global Manager and is focused on helping both students and novice project managers to become aware of the factors that may affect GSD projects. Global Manager was evaluated by means of a quasi-experiment based on a pretest-posttest design, which was used to assess whether the game can increase the players’ awareness and knowledge of the factors that a project manager should consider in GSD projects. The results obtained after evaluating the game were quite satisfactory, as the game taught the players to differentiate between global and distributed software projects and to experience situations that may and do occur in these settings. Furthermore, Global Manager helped improve players' perceptions of the importance of different influencing factors in GSD.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"23 1","pages":"1 - 31"},"PeriodicalIF":2.4,"publicationDate":"2023-04-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44164547","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Evaluation of Submission Limits and Regression Penalties to Improve Student Behavior with Automatic Assessment Systems","authors":"R. Lawrence, Sarah Foss, Tatiana Urazova","doi":"10.1145/3591210","DOIUrl":"https://doi.org/10.1145/3591210","url":null,"abstract":"Objectives. Automatic assessment systems are widely used to provide rapid feedback for students and reduce grading time. Despite the benefits of increased efficiency and improved pedagogical outcomes, an ongoing challenge is mitigating poor student behaviors when interacting with automatic assessment systems including numerous submissions, trial-and-error, and relying on marking feedback for problem solving. These behaviors negatively affect student learning as well as have significant impact on system resources. This research quantitatively examines how utilizing submission policies such as limiting the number of submissions and applying regression penalties can reduce negative student behaviors. The hypothesis is that both submission policies will have a significant impact on student behavior and reduce both the number of submissions and regressions in student performance. The research questions evaluate the impact on student behavior, determine which submission policy is the most effective, and what submission policy is preferred by students. Participants. The study involved two course sections in two different semesters consisting of a total of 224 students at the University of British Columbia, a research-intensive university. The students were evaluated using an automated assessment system in a large third year database course. Study Methods. The two course sections used an automated assessment system for constructing database design diagrams for assignments and exams. The first section had no limits on the number of submissions for both assignments and exams. The second section had limits for the exams but no limits on assignments. On the midterm, participants were randomly assigned to have either a restriction on the total number of submissions or unlimited submissions but with regression penalties if a graded answer was lower than a previous submission. On the final exam, students were given the option of selecting their submission policy. Student academic performance and submission profiles were compared between the course sections and the different submission policies. Findings. Unrestricted use of automatic grading systems results in high occurrence of undesirable student behavior including trial-and-error guessing and reduced time between submissions without sufficient independent thought. Both submission policies of limiting maximum submissions and utilizing regression penalties significantly reduce these behaviors by up to 85%. Overall, students prefer maximum submission limits, and demonstrate improved behavior and educational outcomes. Conclusions. Automated assessment systems when used for larger problems related to design and programming have benefits when deployed with submission restrictions (maximum attempts or regression penalty) for both improved student learning behaviors and to reduce the computational costs for the system. This is especially important for summative assessment but reasonable limits for formative assessme","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"23 1","pages":"1 - 24"},"PeriodicalIF":2.4,"publicationDate":"2023-04-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42792993","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Tracing Participation Beyond Computing Careers: How Women Reflect on Their Experiences in Computing Programs","authors":"Melissa Perez, Patricia Garcia","doi":"https://dl.acm.org/doi/10.1145/3582564","DOIUrl":"https://doi.org/https://dl.acm.org/doi/10.1145/3582564","url":null,"abstract":"<p>Norms and values in computing education are constantly changing as dominant narratives about the role of computing in society evolve over time. Within the current evolving landscape of computing education, researchers and practitioners have advocated for ensuring people from all backgrounds, and particularly women, non-binary, and Black, Indigenous, and Latinx people, are able to participate equitably within the field of computing. Yet, the values of computing educational experiences are narrowly framed within career outcomes, such as securing a career in computing, leaving many important experiences and ways of participating in the field out of the picture. To address this, we conducted reflective interviews with women who participated in broadening participation in computing (BPC) programs to understand their perceptions of computing and how it aligns (or not) with what they value about their experiences in computing learning environments. We investigate the following research questions: (1) How do women who participated in BPC programs describe their perceptions of computing? (2) How do those perceptions align or misalign with the program outcomes they valued? The findings from our study call attention to tensions arising from centering “computing careers” in BPC work and highlight the outcomes of participation valued by the women in our study, such as developing communities and relationships, gaining communication skills, and expanding perspectives on skills computer scientists should possess.</p>","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"36 10","pages":""},"PeriodicalIF":2.4,"publicationDate":"2023-04-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138503014","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Understanding the Impact of Peer Instruction in CS Principles Teacher Professional Development","authors":"Karla Hamlen Mansour, Debbie K. Jackson, Lisa Bievenue, Adam Voight, Nigamanth Sridhar","doi":"https://dl.acm.org/doi/10.1145/3585077","DOIUrl":"https://doi.org/https://dl.acm.org/doi/10.1145/3585077","url":null,"abstract":"<p>In a nationwide initiative to increase computer science Education in K-12, many teachers were recruited to CS teaching positions but without adequate preparation, which can lead to lack of confidence and feelings of isolation [35, 54]. In response to these issues, the purpose of this study was to determine the effectiveness of a newly developed summer professional development program with follow-up support with the goal of improving teacher confidence and informing teacher beliefs about computer science. This training featured peer instruction and an emphasis on computer science principles. It took place over three years, and was used to prepare teachers from over 50 schools across a midwestern state. Using Hierarchical Linear Modeling, data analysis from 57 teachers and 871 students showed that teacher confidence and content knowledge increased significantly, and to a greater extent for those who began with the lowest levels of confidence. Students also made significant knowledge gains from before to after taking the class. Student gains were higher for those whose teachers were more confident in their ability to teach computer science.</p><p></p>","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"36 11","pages":""},"PeriodicalIF":2.4,"publicationDate":"2023-04-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138503013","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Think-Aloud Study of Novice Debugging","authors":"Jacqueline L. Whalley, Amber Settle, Andrew Luxton-Reilly","doi":"10.1145/3589004","DOIUrl":"https://doi.org/10.1145/3589004","url":null,"abstract":"Debugging is a core skill required by programmers, yet we know little about how to effectively teach the process of debugging. The challenges of learning debugging are compounded for novices who lack experience and are still learning the tools they need to program effectively. In this work, we report a case study in which we used a think-aloud protocol to gain insight into the behaviour of three students engaged in debugging tasks. Our qualitative analysis reveals a variety of helpful practices and barriers that limit the effectiveness of debugging. We observe that comprehension, evidence-based activities, and workflow practices all contribute to novice debugging success. Lack of sustained effort, precision, and methodical processes negatively impact debugging effectiveness. We anticipate that understanding how students engage in debugging tasks will aid future work to address ineffective behaviours and promote effective debugging activities.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"23 1","pages":"1 - 38"},"PeriodicalIF":2.4,"publicationDate":"2023-03-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"64073106","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"A Generalized Estimating Equations Approach to Investigate Predictors of Teacher Candidates’ Views of Coding","authors":"B. Belland, Chanmin Kim, A. Y. Zhang, Eunseo Lee","doi":"10.1145/3587163","DOIUrl":"https://doi.org/10.1145/3587163","url":null,"abstract":"This article reports the analysis of data from five different studies to identify predictors of preservice, early childhood teachers’ views of (a) the nature of coding, (b) integration of coding into preschool classrooms, and (c) relation of coding to fields other than computer science (CS). Significant changes in views of coding were predicted by time, prior robot programming experience, and perceptions of the value of coding. Notably, prior programming knowledge and positive perceptions of mathematics predicted decreases in views of coding from pre- to post-survey.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":" ","pages":"1 - 23"},"PeriodicalIF":2.4,"publicationDate":"2023-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45502520","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Teacher Attitudes & Beliefs in Computer Science (T-ABC): Development & Validation of a Teacher Survey Instrument","authors":"Darcy Ronan, D. Cenk Erdil, Dennis Brylow","doi":"https://dl.acm.org/doi/10.1145/3569945","DOIUrl":"https://doi.org/https://dl.acm.org/doi/10.1145/3569945","url":null,"abstract":"<p>Instrument development is an important step towards unlocking the analytical power of teacher attitudes and beliefs towards Computer Science (CS). Teacher dispositions have strong empirical and theoretical ties to teacher motivation, professional choices, and classroom practices. To determine consensus desirable attitudes and beliefs, we analyzed 17 key documents produced by 12 national and international organizations associated with CS and the CS education reform movement. An analysis of 98 relevant coded segments yielded four dispositional targets: an equity orientation, a teacher growth mindset, and key beliefs regarding (career) outcomes and epistemology of CS. Statements crafted for these targets as well as self-efficacy were reviewed through an expert panel (N = 5) and a pilot study (N = 22) before the T-ABC was administered to elementary teachers in a large grant-funded outreach project (N = 772). Psychometric analysis demonstrates high reliability (Cronbach’s alpha = 0.89) and satisfactory extraction and loading onto a three factor model, with CS beliefs, growth mindset, and self-efficacy as major factors. Identification and measurements of teacher dispositions enables further analysis of how teacher beliefs may support or hinder effective practice in CS instruction, how teacher populations may differ, and how identified dispositions may change with exposure to various CS learning experiences.</p>","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"36 12","pages":""},"PeriodicalIF":2.4,"publicationDate":"2023-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138503012","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Barriers and Supports to Offering Computer Science in High Schools: A Case Study of Structures and Agents","authors":"Miranda C. Parker","doi":"https://dl.acm.org/doi/10.1145/3572900","DOIUrl":"https://doi.org/https://dl.acm.org/doi/10.1145/3572900","url":null,"abstract":"<p>In the age of computing, there still exist many schools that do not offer computer science courses. The reason can be esoteric to designers of interventions, curricula, and policies. This study aims to answer the research question: <i>What do school officials perceive as barriers to and supports for offering computer science courses at their school?</i> This article provides a case study of four public high schools in the state of Georgia to better understand the barriers to adopting computer science. Interviews conducted with school principals, counselors, and teachers were thematically analyzed to study each case. The pattern that emerged from these themes is framed by structuration theory, wherein the structures (school policies, resources, and courses) influence the agents (teachers, students, the community), and in turn, the agents influence the structures. This framing provides insights into what attributes can be supports for or barriers against schools offering computer science courses.</p>","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"17 1","pages":""},"PeriodicalIF":2.4,"publicationDate":"2023-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138542683","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Validation of a Spanish-language Version of a Computer Programming Aptitude Test for First-year University Students","authors":"Francisco Vásquez, Juan Felipe Calderón, Federico Meza, Andrea Vásquez","doi":"https://dl.acm.org/doi/10.1145/3579365","DOIUrl":"https://doi.org/https://dl.acm.org/doi/10.1145/3579365","url":null,"abstract":"<p>There is increasing interest in computer science and computing bachelor programs due to the growing importance of technology in the globalized world. Thus, as higher education institutions strive to serve a diverse student demographic, it is salient to gauge their programming abilities to improve guidance on learning processes regarding their initial knowledge state. Despite the availability of certain instruments to measure student programming skills, these are traditionally aimed at younger populations and do not accurately discriminate the different levels of ability among university students. This article introduces a translation into Spanish and validation of an existing English-language aptitude test for computing jobs that can be used to measure the programming abilities of students with no prior experience in the field. Following a cyclic research methodology, two iterations were carried out in this article. First, the aforementioned test was translated and validated via expert judgment and focus groups, in which certain items were removed subsequent to a quantitative analysis. The resultant instrument underwent a second validation using a larger population of students. Analysis conducted after the second iteration showed this instrument to deliver good internal consistency, good difficulty and discrimination indices, and a moderate correlation with the grades of the midterm exam of a programming course undertaken by first year engineering students. This work contributes to both increasing the number of tests available in the Spanish language with which to assess programming abilities, as well as to the broader literature regarding test adaptation, translation and validation.</p>","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"36 14","pages":""},"PeriodicalIF":2.4,"publicationDate":"2023-03-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"138503001","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}