ACM Transactions on Computing Education最新文献

Evaluating ChatGPT-4 Vision on Brazil’s National Undergraduate Computer Science Exam 在巴西全国本科计算机科学考试中评估 ChatGPT-4 Vision

IF 2.4 3区工程技术

ACM Transactions on Computing Education Pub Date : 2024-06-20 DOI: 10.1145/3674149

Nabor C. Mendonça

{"title":"Evaluating ChatGPT-4 Vision on Brazil’s National Undergraduate Computer Science Exam","authors":"Nabor C. Mendonça","doi":"10.1145/3674149","DOIUrl":"https://doi.org/10.1145/3674149","url":null,"abstract":"The recent integration of visual capabilities into Large Language Models (LLMs) has the potential to play a pivotal role in science and technology education, where visual elements such as diagrams, charts, and tables are commonly used to improve the learning experience. This study investigates the performance of ChatGPT-4 Vision, OpenAI’s most advanced visual model at the time the study was conducted, on the Bachelor in Computer Science section of Brazil’s 2021 National Undergraduate Exam (ENADE). By presenting the model with the exam’s open and multiple-choice questions in their original image format and allowing for reassessment in response to differing answer keys, we were able to evaluate the model’s reasoning and self-reflecting capabilities in a large-scale academic assessment involving textual and visual content. ChatGPT-4 Vision significantly outperformed the average exam participant, positioning itself within the top 10 best score percentile. While it excelled in questions that incorporated visual elements, it also encountered challenges with question interpretation, logical reasoning, and visual acuity. A positive correlation between the model’s performance in multiple-choice questions and the performance distribution of the human participants suggests multimodal LLMs can provide a useful tool for question testing and refinement. However, the involvement of an independent expert panel to review cases of disagreement between the model and the answer key revealed some poorly constructed questions containing vague or ambiguous statements, calling attention to the critical need for improved question design in future exams. Our findings suggest that while ChatGPT-4 Vision shows promise in multimodal academic evaluations, human oversight remains crucial for verifying the model’s accuracy and ensuring the fairness of high-stakes educational exams. The paper’s research materials are publicly available at https://github.com/nabormendonca/gpt-4v-enade-cs-2021.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"24 1","pages":""},"PeriodicalIF":2.4,"publicationDate":"2024-06-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141501605","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Understanding Informatics in Continuing Vocational Education and Training Data in Germany 了解德国继续职业教育和培训数据中的信息学

IF 2.4 3区工程技术

ACM Transactions on Computing Education Pub Date : 2024-06-19 DOI: 10.1145/3665932

Jens Dörpinghaus, Johanna Binnewitt, David Samray, Kristine Hein

{"title":"Understanding Informatics in Continuing Vocational Education and Training Data in Germany","authors":"Jens Dörpinghaus, Johanna Binnewitt, David Samray, Kristine Hein","doi":"10.1145/3665932","DOIUrl":"https://doi.org/10.1145/3665932","url":null,"abstract":"Objectives The purpose of this study is to reveal the importance of informatics in continuing vocational education in Germany. The labour market is a field with diverse data structures and multiple applications, for example connecting jobseekers and trainings or jobs. The labour market heavily relies on vocational education and training and advanced vocational qualification to meet challenges, e.g. digitalization.Study Methods Since continuing vocational education and training (CVET) is a structurally important lever for the digital transformation of work, this article presents a methodological procedure for content analysis that provides information about the significance of computer science in unregulated continuing education offerings and in formal continuing education regulations.Findings The question of the extent to which continuing education programs include informaticss topics is investigated, assuming that they can be found in continuing education as cross-cutting topics in a wide variety of thematic contexts. Our results indicating the need for training in computing education. At the same time, computing education offers the highest share of unregulated CVET programs. This could reflect the fact that training and further education regulations in Germany are designed open to technology.Conclusions We present a novel and unique approach to analyze the importance of informatics and digitalization in CVET advertisements and official regulations for the same.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"47 1","pages":""},"PeriodicalIF":2.4,"publicationDate":"2024-06-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141501471","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Intent and Extent: Computer Science Concepts and Practices in Integrated Computing 意图和范围：综合计算中的计算机科学概念与实践

IF 2.4 3区工程技术

ACM Transactions on Computing Education Pub Date : 2024-05-14 DOI: 10.1145/3664825

Lauren E. Margulieux, Yin-Chan Liao, Erin Anderson, Miranda C. Parker, Brendan D. Calandra

{"title":"Intent and Extent: Computer Science Concepts and Practices in Integrated Computing","authors":"Lauren E. Margulieux, Yin-Chan Liao, Erin Anderson, Miranda C. Parker, Brendan D. Calandra","doi":"10.1145/3664825","DOIUrl":"https://doi.org/10.1145/3664825","url":null,"abstract":"Integrated computing curricula combine learning objectives in computing with those in another discipline, like literacy, math, or science, to give all students experience with computing, typically before they must decide whether to take standalone CS courses. One goal of integrated computing curricula is to provide an accessible path to an introductory computing course by introducing computing concepts and practices in required courses. This study analyzed integrated computing curricula to determine which CS practices and concepts are taught, how extensively the curricula are taught, and, by extension, how they might prepare students for later computing courses. The authors conducted a content analysis to examine primary and lower secondary (i.e., K-8) curricula that are taught in non-CS classrooms, have explicit CS learning objectives (i.e., CS+X), and that took 5+ hours to complete. Lesson plans, descriptions, and resources were scored based on frameworks developed from the K-12 CS Framework, including programming concepts, non-programming CS concepts, and CS practices. The results found that curricula most extensively taught introductory concepts and practices, such as sequences, and rarely taught more advanced content, such as conditionals. Students who engage with most of these curricula would have no experience working with fundamental concepts, like variables, operators, data collection or storage, or abstraction in the context of a program. While this focus might be appropriate for integrated curricula, it has implications for the prior knowledge that students should be expected to have when starting standalone computing courses.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"18 1","pages":""},"PeriodicalIF":2.4,"publicationDate":"2024-05-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140928811","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Creating Apps for Community and Social Good: Preliminary Learning Outcomes from a Middle School Computer Science Curriculum 为社区和社会公益创建应用程序：初中计算机科学课程的初步学习成果

IF 2.4 3区工程技术

ACM Transactions on Computing Education Pub Date : 2024-04-15 DOI: 10.1145/3658674

Lijun Ni, Gillian Bausch, Elizabeth Thomas-Cappello, Fred Martin, Bernardo Feliciano

{"title":"Creating Apps for Community and Social Good: Preliminary Learning Outcomes from a Middle School Computer Science Curriculum","authors":"Lijun Ni, Gillian Bausch, Elizabeth Thomas-Cappello, Fred Martin, Bernardo Feliciano","doi":"10.1145/3658674","DOIUrl":"https://doi.org/10.1145/3658674","url":null,"abstract":"This study examined student learning outcomes from a middle school computer science (CS) curriculum developed through a researcher-practitioner partnership (RPP) project. The curriculum is based on students creating mobile apps that serve community and social good. We collected two sets of data from 294 students in three urban districts: (1) pre- and post- survey responses on their learning experiences and attitudes toward learning CS and creating community-serving apps; (2) the apps created by those students. The analysis of student apps indicated that students were able to create basic apps that connected with their personal interests, life experiences, school community, and the larger society. Students were significantly more confident in coding and creating community-focused apps after completing the course, regardless of gender, race/ethnicity, and grade. However, their interest in solving coding problems and continuing to learn CS decreased afterward. Analyses of students’ attitudes by gender, grade, and race/ethnicity showed significant differences among students in some groups. Seventh grade students rated more positively on their attitudes than eighth graders. Students identifying with different race/ethnicity groups indicated significantly different attitudes, especially students identifying as Southeast Asian, Black/African American, and Hispanic/Latino. Self-identified male students also reported stronger interest and more positive attitudes overall than self-identified female students. Students also reported positive experiences in learning how to create real apps serving their community, while there were disparities in their experiences with coding in general and some of the instructional tools used in the class.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"48 1","pages":""},"PeriodicalIF":2.4,"publicationDate":"2024-04-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140586797","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Doing and Defining Interdisciplinarity in Undergraduate Computing 在本科生计算机学习中实践和定义跨学科性

IF 2.4 3区工程技术

ACM Transactions on Computing Education Pub Date : 2024-04-04 DOI: 10.1145/3654676

Kaitlin N. S. Newhouse, Kathleen J. Lehman, Annie M. Wofford, Michelle Sendowski

{"title":"Doing and Defining Interdisciplinarity in Undergraduate Computing","authors":"Kaitlin N. S. Newhouse, Kathleen J. Lehman, Annie M. Wofford, Michelle Sendowski","doi":"10.1145/3654676","DOIUrl":"https://doi.org/10.1145/3654676","url":null,"abstract":"Interdisciplinarity has been touted as a means to recruit more racially and gender diverse students to computing. In this explanatory sequential mixed-methods study, we investigated demographic characteristics among a sample of undergraduate students pursuing interdisciplinary computing major and minor combinations at 15 institutions in the United States who completed a survey at the end of their introductory course. Descriptive analyses of responses to this survey of introductory computing students revealed that enrollment in interdisciplinary combinations was limited and did not appear to disproportionately attract women or Black/African American, Latine, Indigenous, and Multiracial students. We then conducted a directed content analysis of departmental websites to examine the language and policies that may preclude or encourage students to pursue interdisciplinary computing major and minor combinations. Findings revealed that departmental offerings of such programs were limited, and, among those that did offer such programs, communication about their goals and requirements was often lacking. Implications for research and practice, especially as they pertain to efforts to broaden participation in computing, are discussed.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"58 1","pages":""},"PeriodicalIF":2.4,"publicationDate":"2024-04-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140586879","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

False Positives and Deceptive Errors in SQL Assessment: A Large-scale Analysis of Online Judge Systems SQL 评估中的假阳性和欺骗性错误：在线评判系统的大规模分析

IF 2.4 3区工程技术

ACM Transactions on Computing Education Pub Date : 2024-03-28 DOI: 10.1145/3654677

Jinshui Wang, Shuguang Chen, Zhengyi Tang, Pengchen Lin, Yupeng Wang

{"title":"False Positives and Deceptive Errors in SQL Assessment: A Large-scale Analysis of Online Judge Systems","authors":"Jinshui Wang, Shuguang Chen, Zhengyi Tang, Pengchen Lin, Yupeng Wang","doi":"10.1145/3654677","DOIUrl":"https://doi.org/10.1145/3654677","url":null,"abstract":"Online Judge Systems (OJSs) play a crucial role in evaluating SQL programming skills. However, OJSs may not accurately evaluate students’ queries due to the error-detection capabilities of test sets are insufficient, resulting in false positives that can mislead students and hinder their learning. This study analyzes a large-scale OJS’s evaluation dataset and identifies more than 110,000 (1.94%) false positive queries. It also validates existing SQL error categorization and reveals a new type of logical error called deceptive error, which occurs when students construct queries that pass specific test cases but fail to solve the actual problem. This type of error has been overlooked in previous research and can provide new insights into how to improve OJSs by enhancing test cases and feedback. This study contributes to the understanding of assessment and evaluation practices and processes in programming education, particularly the contribution that OJSs make to student learning and to course, staff and institutional development. It also suggests error prevention and detection techniques that can improve the effectiveness and fairness of OJSs in programming education and competitions.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"42 1","pages":""},"PeriodicalIF":2.4,"publicationDate":"2024-03-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140325335","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Introduction to the Special Issue on Situating the Intersectional Experiences of Black Girls and Women in Computing & Technology 黑人女孩和妇女在计算机与技术领域的交叉经历特刊简介

IF 2.4 3区工程技术

ACM Transactions on Computing Education Pub Date : 2024-03-05 DOI: 10.1145/3648478

Jakita O. Thomas, Quincy K. Brown, Jamika Burge

引用次数: 0

Debugging Pathways: Open-Ended Discrepancy Noticing, Causal Reasoning, and Intervening 调试路径：开放式差异注意、因果推理和干预

IF 2.4 3区工程技术

ACM Transactions on Computing Education Pub Date : 2024-03-05 DOI: 10.1145/3650115

David DeLiema, Jeffrey K. Bye, Vijay Marupudi

{"title":"Debugging Pathways: Open-Ended Discrepancy Noticing, Causal Reasoning, and Intervening","authors":"David DeLiema, Jeffrey K. Bye, Vijay Marupudi","doi":"10.1145/3650115","DOIUrl":"https://doi.org/10.1145/3650115","url":null,"abstract":"Learning to respond to a computer program that is not working as intended is often characterized as finding a singular bug causing a singular problem. This framing underemphasizes the wide range of ways that students and teachers could notice discrepancies from their intention, propose causes of those discrepancies, and implement interventions. Weaving together a synthesis of the existing research literature with new multimodal interaction analyses of teacher-student conversations during coding, we propose a framework for debugging that foregrounds this open-endedness. We use the framework to structure an analysis of three naturalistic debugging situations (with U.S. 5th–10th graders) that range from solo debugging to collaborative discourse. We argue that a broken computer program is a polysemous object through which teachers and students actively and publicly notice, reason about, and negotiate different debugging pathways. We document students and teachers improvisationally altering a debugging pathway, justifying a particular pathway, and outwardly discussing competing pathways. This paper provides a framework for structuring debugging pedagogy to be less about scaffolding a student toward a specific pathway to a fix, and more about exploring multiple possible pathways and judging the (learning) value of various routes.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"86 1","pages":""},"PeriodicalIF":2.4,"publicationDate":"2024-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140045393","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

Professional Development in Computational Thinking: A Systematic Literature Review 计算思维的专业发展：系统文献综述

IF 2.4 3区工程技术

ACM Transactions on Computing Education Pub Date : 2024-03-05 DOI: 10.1145/3648477

Alejandro Espinal, Camilo Vieira, Alejandra J. Magana

{"title":"Professional Development in Computational Thinking: A Systematic Literature Review","authors":"Alejandro Espinal, Camilo Vieira, Alejandra J. Magana","doi":"10.1145/3648477","DOIUrl":"https://doi.org/10.1145/3648477","url":null,"abstract":"This paper presents a systematic literature review of professional development programs in Computational Thinking. Computational thinking (CT) has emerged as an essential set of skills that everyone should develop to participate in a global society. However, there were no pre-service or in-service teacher programs to integrate CT into the K-12 classrooms until very recently. Thus, it is important to identify how educators and researchers address the challenges to prepare the next generation of students and what gaps persist in the current literature. We review existing work in this field from two perspectives: First, we analyze the learning outcomes, assessment methods, pedagogical approaches, and pedagogical tools used in the Professional Development programs in CT. Second, we examine how these programs assess the teachers’ knowledge and skills as outcomes. We used the technological pedagogical and content knowledge (TPACK) framework to characterize existing literature and identify possible gaps in the preparation of pre-service and in-service teachers in CT. Our results suggest that: (1) existing evidence is limited to developed countries; (2) many studies are only focusing on teachers understanding the concepts but do not explore how the participants evaluate or create learning activities; (3) no studies look into classroom observations as part of the program, which limits our understanding to how these programs work; and (4) most programs use block-based programming languages as the tool to develop student computational thinking. While block-based programming languages are used for introductory training programs, students are often expected to transfer their learning to more professional programming languages.","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"2013 1","pages":""},"PeriodicalIF":2.4,"publicationDate":"2024-03-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140045631","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0

How Do First-Year Engineering Students’ Emotions Change While Working on Programming Problems? 工程专业一年级学生在解决编程问题时的情绪如何变化？

IF 2.4 3区工程技术

ACM Transactions on Computing Education Pub Date : 2024-02-09 DOI: 10.1145/3643865

Zahra Atiq, Rakhi Batra

{"title":"How Do First-Year Engineering Students’ Emotions Change While Working on Programming Problems?","authors":"Zahra Atiq, Rakhi Batra","doi":"10.1145/3643865","DOIUrl":"https://doi.org/10.1145/3643865","url":null,"abstract":"Emotions are a complex multi-faceted phenomenon. To assess the complexity of emotions from different facets, multi-modal approaches are necessary. However, multi-modal approaches are rarely used for assessing emotions, especially in the context of computer programming. This study adopts a multi-modal approach to understand the changes in students’ perception of emotions before and after working on programming problems. Understanding these changes in students’ perceptions may enable educators to devise interventions that help students adjust their perceptions and regulate their emotions as per their skills. We conducted a one-on-one programming session and retrospective think-aloud interview with seventeen students from an introductory programming course. During the programming session, students filled surveys and performed four programming tasks. While working on these tasks, students’ eye gaze, video of face and screen, and electrodermal activity data were also collected using a non-invasive device. The data collection for this study was multi-modal, with a mix of both qualitative and quantitative data collection methods. Data analysis was primarily qualitative, with additional triangulation of qualitative and biometric data for select exemplars. The findings of this study suggest that students experience changes in emotions because of many reasons, for instance, they encountered repeated errors, they set high standards for their performance, or they could not manage time. For some students, negative emotions changed to positive emotions when they solved errors without any external help or achieved more than what they expected going into the task. Moreover, the triangulation of qualitative and biometric data of two participants provides a fine-grained analysis of their emotions and behaviors and confirmed the change in the perception of their emotions while performing the programming tasks.Emotions, Multi-modal data, and Programming","PeriodicalId":48764,"journal":{"name":"ACM Transactions on Computing Education","volume":"94 1","pages":""},"PeriodicalIF":2.4,"publicationDate":"2024-02-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139753444","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":3,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

引用次数: 0