{"title":"Improving Student Motivation by Ungrading","authors":"Scott Spurlock","doi":"10.1145/3545945.3569747","DOIUrl":"https://doi.org/10.1145/3545945.3569747","url":null,"abstract":"Recent interest in alternative grading strategies has been increasing in the Computer Science Education community. The umbrella term ungrading has been used to refer to a variety of practices that de-emphasize numeric grades. In this paper we present the results of implementing an ungrading scheme that eliminates numeric grades, allows resubmission of assignments, and encourages student input into their final assigned letter grade. We administered surveys measuring student attitudes and motivation at the start and end of three different upper level Computer Science elective courses using the new grading scheme and found a significant increase in students' feelings of intrinsic goal orientation (valuing coursework for its own sake), self-efficacy (feeling able to be successful), and control of learning (taking responsibility for their own learning). We observed that, given the opportunity, most students chose to redo only a small number of assignments, and most students requested final grades within a half-letter of the instructor's estimate. Overall, compared with prior iterations of the courses that were graded traditionally, the final grade point average did not significantly increase, while students' reported level of effort did significantly increase. Comments on post-course surveys indicate that students liked the new grading scheme, and they reported improved learning and reduced anxiety.","PeriodicalId":371326,"journal":{"name":"Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1","volume":"57 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121403015","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
James Skripchuk, Neil Bennett, Jeffrey Zhang, Eric Li, T. Price
{"title":"Analysis of Novices' Web-Based Help-Seeking Behavior While Programming","authors":"James Skripchuk, Neil Bennett, Jeffrey Zhang, Eric Li, T. Price","doi":"10.1145/3545945.3569852","DOIUrl":"https://doi.org/10.1145/3545945.3569852","url":null,"abstract":"Web-based help-seeking -- finding and utilizing websites to solve a problem -- is a critical skill during programming in both professional and academic settings. However, little work has explored how students, especially novices, engage in web-based help-seeking during programming, or what strategies they use and barriers they face. This study begins to investigate these questions through analysis of students' web-search behaviors during programming. We collected think-aloud, screen recording, and log data as students completed a challenging programming task. Students were encouraged to use the web for help when needed, as if in an internship. We then qualitatively analyzed the data to address three research questions: 1) What events motivate students to use web search? 2) What strategies do students employ to search for, select, and learn from web pages? 3) What barriers do students face in web search, and when do they arise? Our results suggest that that novices use a variety of web-search strategies -- some quite unexpected -- with varying degrees of success, suggesting that web search can be a challenging skill for novice programmers. We discuss how these results inform future research and pedagogy focused on how to support students in effective web search.","PeriodicalId":371326,"journal":{"name":"Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1","volume":"88 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"123759250","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Elliot Varoy, K. Lee, Andrew Luxton-Reilly, Nasser Giacaman
{"title":"Draw a Computing Student: Facilitating Semi-Structured Interviews Through Drawing","authors":"Elliot Varoy, K. Lee, Andrew Luxton-Reilly, Nasser Giacaman","doi":"10.1145/3545945.3569795","DOIUrl":"https://doi.org/10.1145/3545945.3569795","url":null,"abstract":"Qualitative research methodologies that collect data through interviews, focus groups and ethnographic studies, are valuable approaches to understand the educational experience. Participant drawings can be used as prompts to access implicit understandings and have been shown to contribute to effective data collection in related fields, but are not widely used in computing education research. We report on a case study in which sixteen New Zealand primary and secondary school digital technologies teachers were interviewed and asked to draw their ideal computing student. This case study illustrates how drawings can be used to enhance semi-structured interviews in the context of computing education. The contributions made here come in the form of a case study demonstrating the draw-a-person method in a computing education context, along with recommendations for researchers considering applying this method in their work.","PeriodicalId":371326,"journal":{"name":"Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1","volume":"5 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126813918","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Stream Your Exam to the Course Staff: Asynchronous Assessment via Student-Recorded Code Trace Videos","authors":"Rachel S. Lim, J. Politz, Mia Minnes","doi":"10.1145/3545945.3569803","DOIUrl":"https://doi.org/10.1145/3545945.3569803","url":null,"abstract":"Now that students in our introductory object-oriented programming course are more familiar with Zoom and screen sharing, we consider novel assessments that leverage these tools. We developed a style of assessment where students submit a recorded screencast of them tracing a submitted program. We describe the design of the assessment and tracing prompts, and we report on an analysis of 59 submitted student videos. Our findings include common mistakes and aspects of student understanding that were expressed in the video but not in the text and code submitted by students. For example, we observed different strategies that yielded the same correct trace of a loop, as well as incorrect traces of students' own correct recursive programs. These guide us towards ways to refine the assessment and prompts in future iterations.","PeriodicalId":371326,"journal":{"name":"Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1","volume":"21 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126149055","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Towards Finding the Missing Pieces to Teach Secure Programming Skills to Students","authors":"Majed Almansoori, Jessica Lam, Elias Fang, Adalbert Gerald Soosai Raj, Rahul Chatterjee","doi":"10.1145/3545945.3569730","DOIUrl":"https://doi.org/10.1145/3545945.3569730","url":null,"abstract":"Research efforts tried to expose students to security topics early in the undergraduate CS curriculum. However, such efforts are rarely adopted in practice and remain less effective when it comes to writing secure code. In our prior work [18], we identified key issues with the how students code and grouped them into six themes: (a) Knowledge of C, (b) Understanding compiler and OS messages, (c) Utilization of resources, (d) Knowledge of memory, (e) Awareness of unsafe functions, and (f) Understanding of security topics. In this work, we aim to understand students' knowledge about each theme and how that knowledge affects their secure coding practices. Thus, we propose a modified SOLO taxonomy for the latter five themes. We apply the taxonomy to the coding interview data of 21 students from two US R1 universities. Our results suggest that most students have limited knowledge of each theme. We also show that scoring low in these themes correlates with why students fail to write secure code and identify possible vulnerabilities.","PeriodicalId":371326,"journal":{"name":"Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1","volume":"2014 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127479209","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Discovering and Quantifying Misconceptions in Formal Methods Using Intelligent Tutoring Systems","authors":"Marko Schmellenkamp, Alexandra Latys, T. Zeume","doi":"10.1145/3545945.3569806","DOIUrl":"https://doi.org/10.1145/3545945.3569806","url":null,"abstract":"In this paper we advocate the study of misconceptions in the formal methods domain by integrating quantitative and qualitative methods. In this domain, so far, misconceptions have mostly been studied with qualitative methods, typically via interviews with less than 20 subjects. We discuss workflows for (1) determining the commonness of qualitatively established misconceptions by quantitative means; and for (2) the initial discovery of misconceptions by quantitative methods followed by qualitative assessments. Parts of these workflows are then applied to a data set for exercises on logical modeling from the intelligent tutoring system Iltis with > 250 data points for many of the exercises. We analyze the data in order to (1) determine the commonness of qualitativelyidentified misconceptions on modeling in propositional logic; and to (2) discover typical mistakes in modeling in propositional logic, modal logic, and first-order logic.","PeriodicalId":371326,"journal":{"name":"Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127434747","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Putting a Context in Context: Investigating the Context of Pencil Puzzles in Multiple Academic Environments","authors":"Z. Butler, Ivona Bezáková, A. Brilliantova","doi":"10.1145/3545945.3569848","DOIUrl":"https://doi.org/10.1145/3545945.3569848","url":null,"abstract":"The use of a well-chosen context for course assignments is widely regarded as motivating for students. However, it is challenging to study the utility of bringing a particular context to computing courses across different types of institutions and student demo- graphics. This is especially true in introductory computing since courses vary widely, for example, in topic order and depth of coverage. In this experience report, we present our approach to, and lessons learned from, studying the efficacy of a specific context for introductory computing assignments across a variety of environments. We focus on the context of pencil puzzles (puzzles like Sudoku or crosswords, designed to be solved on paper using a pencil) and the deployment and fit of pencil-puzzle-based assignments across different institutions' introductory curricula. We describe our overall process, including recruitment of instructors from a variety of institutions, development and deployment of assignments, and collection of student grade and survey data (including all necessary approvals). By design, we did not use the same assignment at each university, since we aimed to study the underlying context rather than a specific assignment, while also establishing the adoptability of the context to different circumstances. We discuss the heterogeneity of the resulting data set, how we chose to analyze it, and what conclusions can (and cannot) be drawn from such data. We conclude with lessons learned from this experience, with the hopes that they can help others who wish to propagate their innovations and study them in diverse situations.","PeriodicalId":371326,"journal":{"name":"Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131957083","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"The Programming Exercise Markup Language: Towards Reducing the Effort Needed to Use Automated Grading Tools","authors":"D. S. Mishra, S. Edwards","doi":"10.1145/3545945.3569734","DOIUrl":"https://doi.org/10.1145/3545945.3569734","url":null,"abstract":"Automated programming assignment grading tools have become integral to CS courses at introductory as well as advanced levels. However such tools have their own custom approaches to setting up assignments and describing how solutions should be tested, requiring instructors to make a significant learning investment to begin using a new tool. In addition, differences between tools mean that initial investment must be repeated when switching tools or adding a new one. Worse still, tool-specific strategies further reduce the ability of educators to share and reuse their assignments. This paper describes an early experiences with PEML, the Programming Exercise Markup Language, which provides an easy to use, instructor friendly approach for writing programming assignments. Unlike tool-oriented data interchange formats, PEML is designed to provide a human friendly authoring format that has been developed to be intuitive, expressive and not be a technological or notational barrier to instructors. We describe the design and implementation of PEML, both as a programming library and also a public-access web microservice that provides full parsing and rendering capabilities for easy integration into any tools or scripting libraries. We also describe experiences using PEML to describe a full range of programming assignments, laboratory exercises, and small coding questions of varying complexity in demonstrating the practicality of the notation. The aim is to develop PEML as a community resource to reduce the barriers to entry for automated assignment tools while widening the scope of programming assignment sharing and reuse across courses and institutions.","PeriodicalId":371326,"journal":{"name":"Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1","volume":"33 25","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131806025","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Vidushi Ojha, Christopher Perdriau, Brent Lagesse, Colleen M. Lewis
{"title":"Computing Specializations: Perceptions of AI and Cybersecurity Among CS Students","authors":"Vidushi Ojha, Christopher Perdriau, Brent Lagesse, Colleen M. Lewis","doi":"10.1145/3545945.3569782","DOIUrl":"https://doi.org/10.1145/3545945.3569782","url":null,"abstract":"Artificial intelligence (AI) and cybersecurity are in-demand skills, but little is known about what factors influence computer science (CS) undergraduate students' decisions on whether to specialize in AI or cybersecurity and how these factors may differ between populations. In this study, we interviewed undergraduate CS majors about their perceptions of AI and cybersecurity. Qualitative analyses of these interviews show that students have narrow beliefs about what kind of work AI and cybersecurity entail, the kinds of people who work in these fields, and the potential societal impact AI and cybersecurity may have. Specifically, students tended to believe that all work in AI requires math and training models, while cybersecurity consists of low-level programming; that innately smart people work in both fields; that working in AI comes with ethical concerns; and that cybersecurity skills are important in contemporary society. Some of these perceptions reinforce existing stereotypes about computing and may disproportionately affect the participation of students from groups historically underrepresented in computing. Our key contribution is identifying beliefs that students expressed about AI and cybersecurity that may affect their interest in pursuing the two fields and may, therefore, inform efforts to expand students' views of AI and cybersecurity. Expanding student perceptions of AI and cybersecurity may help correct misconceptions and challenge narrow definitions, which in turn can encourage participation in these fields from all students.","PeriodicalId":371326,"journal":{"name":"Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1","volume":"33 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132837269","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Detecting the Reasons for Program Decomposition in CS1 and Evaluating Their Impact","authors":"Charis Charitsis, C. Piech, John C. Mitchell","doi":"10.1145/3545945.3569763","DOIUrl":"https://doi.org/10.1145/3545945.3569763","url":null,"abstract":"Decomposition is considered one of the four cornerstones of computational thinking, which is essential to software development [36]. It requires the ability to assess a problem at a high level, develop a strategy to combat it, and then design a solution. Our study focuses on the metacognitive aspect of decomposition. We try to understand the learner's thought process and, specifically, what makes the novice programmer decide to break down a function. Researchers have studied decomposition in introductory programming courses through guided experiments, case studies, and surveys[23,37]. In this work, we follow a different, more scalable approach. We develop an automated system to analyze 45,000 code snapshots from 168 students for a challenging CS1 programming assignment, detect the pivotal moments when they decide to decompose their programs, and identify what drives their decisions from the code. We then classify the students and study the relationship between the different categories, the code complexity, and the time to derive the final solution. We evaluate the impact of decomposition on the student's performance in the assignment and the course exams. Finally, we discuss the implications of our results for computing education.","PeriodicalId":371326,"journal":{"name":"Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1","volume":"13 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-03-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130819130","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}