Research Conference 2022: Reimagining assessment: Proceedings and program最新文献

{"title":"An innovative method for teachers to formatively assess writing online","authors":"Sandra Heldsinger, Stephen M. Humphry","doi":"10.37517/978-1-74286-685-7-1","DOIUrl":"https://doi.org/10.37517/978-1-74286-685-7-1","url":null,"abstract":"Assessment is an integral component of effective teaching and a teacher’s professional judgement influences all routine aspects of their work. In the last 20 years, there has been considerable work internationally to support teachers in using assessment to improve student learning. However, there is a pressing issue that impedes teacher professional judgement being exploited to its full potential. The issue relates to teacher assessments in the context of extended performances such as essays and arises from the complexity of obtaining reliable or consistent teacher assessments of students’ work. Literature published in the United States, England and Australia details evidence of low reliability and bias in teacher assessments. As a result, despite policymakers’ willingness to consider making greater use of teachers’ judgements in summative assessment, and thus provide for greater parity of esteem between teachers’ assessment and standardised testing, few gains have been made. While low reliability of scoring is a pressing issue in contexts where the data are used for summative purposes, it also an issue for formative assessment. Inaccurate assessment necessarily impedes the effectiveness of any follow-up activity, and hence the effectiveness of formative assessment. In this session, Dr Sandy Heldsinger and Dr Stephen Humphry will share their research of writing assessment and explain how their research has led to the development of an innovative assessment process that provides the advantages of rubrics, comparative judgements and automated marking with few of the disadvantages.","PeriodicalId":191950,"journal":{"name":"Research Conference 2022: Reimagining assessment: Proceedings and program","volume":"41 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131314980","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Sharing and securing learners’ performance standards across schools","authors":"R. Kimbell","doi":"10.37517/978-1-74286-685-7-6","DOIUrl":"https://doi.org/10.37517/978-1-74286-685-7-6","url":null,"abstract":"Assessing learners’ performance makes very different demands upon teachers depending on the purpose and the context of the assessment. But common to all assessment is some sense of what ‘quality’ looks like. Most often teachers engage in formative assessments in the classroom, and the familiar standards of the classroom are adequate for this purpose. However if teachers are to undertake external, nationally regulated assessment then some sense of a national standard of quality is required. But there are very limited mechanisms by which teachers can acquire this understanding, so they use their best judgement, and standards vary from school to school not because anyone is attempting to cheat the system but simply because they cannot know what the real national standard is. It is for this reason that regulated examination bodies follow some process such as the following from the State Examinations Commission (SEC) in Ireland. ‘… teacher estimated marks will be subjected to an in-school alignment process and later a national standardisation process’. (SEC, 2021). How much simpler it would all be if teachers had – as a matter of normal practice – access to, and familiarity with, work from a national sample of schools, not just their own classroom. Adaptive Comparative Judgement (ACJ) is an online assessment tool that has been used for some years, principally as a formative tool for learners (e.g. Bartholomew et al., 2018; 2019). This presentation reports on a study of the new ACJ Steady State tool from the same stable. The purpose of the new tool is to solve the problem of variable standards across schools by enabling teachers to make paired judgements of work from multiple schools and thereby evolve and agree standards of performance beyond their own school. The current study is operating in Ireland with a group of schools, a university, and the SEC. The anticipated outcomes include 1) better consistency of performance standards across schools in the research group and 2) greater understanding of and confidence in assessment judgements by the teachers. If ACJ has proved to be a powerful formative assessment tool for learners, ACJ Steady State is designed to be a formative assessment tool for teachers, helping to inform and support their assessment judgements.","PeriodicalId":191950,"journal":{"name":"Research Conference 2022: Reimagining assessment: Proceedings and program","volume":"114 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124068851","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Teachers’ assessment literacy and design competence framework","authors":"Sladana Krstic, S. Manlove, Sarah Richardson","doi":"10.37517/978-1-74286-685-7-8","DOIUrl":"https://doi.org/10.37517/978-1-74286-685-7-8","url":null,"abstract":"The skills demonstrated by a proficient reader are not easy to untangle. Current research acknowledges that reading comprehension is a highly complex area of ability, one that needs to be understood as the coordination of several integrated processes. Using example test questions and data, this presentation explores how assessment can help us make sense of reading comprehension in a way that curricula and commonly used teaching strategies cannot. Assessment is evidence that informs us about the skills involved in the reading process, how they relate to each other, and how they develop in complexity. When assessment is understood in this way, as much more than a tool to compare a student’s ability with that of their peers, it can be used to identify the actual skills individual students are consolidating and which specific steps will help their development.","PeriodicalId":191950,"journal":{"name":"Research Conference 2022: Reimagining assessment: Proceedings and program","volume":"28 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"124091796","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Making learning visible: Moving from nouns to verbs","authors":"D. DeBacker","doi":"10.37517/978-1-74286-685-7-3","DOIUrl":"https://doi.org/10.37517/978-1-74286-685-7-3","url":null,"abstract":"This presentation focuses on the need to make learning visible for all constituents – learners, educators, employers, policymakers and future employees – with a focus on the work of the Center for Certification and Competency-Based Education (C3Be) at the University of Kansas (USA). As K–12 and higher education move to a post-pandemic world, it’s imperative we find different ways to identify learner outcomes and evaluate learning. By moving from nouns to verbs; by moving from course titles to competencies; by moving from transcripts to learner outcomes, learning becomes visible. International educator and founding director of C3Be, Dr Diane DeBacker shows how learning is made visible through a proven process of mapping learning outcomes, assessing both new and prior learning using innovative technology, issuing stackable micro-credentials, and working with policymakers and employers to meet workforce demands.","PeriodicalId":191950,"journal":{"name":"Research Conference 2022: Reimagining assessment: Proceedings and program","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130752931","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Assessing reading: How assessment can be used to target teaching and enhance understanding of reading comprehension","authors":"Sandra Knowles","doi":"10.37517/978-1-74286-685-7-7","DOIUrl":"https://doi.org/10.37517/978-1-74286-685-7-7","url":null,"abstract":"The skills demonstrated by a proficient reader are not easy to untangle. Current research acknowledges that reading comprehension is a highly complex area of ability, one that needs to be understood as the coordination of several integrated processes. Using example test questions and data, this presentation explores how assessment can help us make sense of reading comprehension in a way that curricula and commonly used teaching strategies cannot. Assessment is evidence that informs us about the skills involved in the reading process, how they relate to each other, and how they develop in complexity. When assessment is understood in this way, as much more than a tool to compare a student’s ability with that of their peers, it can be used to identify the actual skills individual students are consolidating and which specific steps will help their development.","PeriodicalId":191950,"journal":{"name":"Research Conference 2022: Reimagining assessment: Proceedings and program","volume":"9 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126486182","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Using PAT data to inform teaching and learning","authors":"Marc Kralj, R. Felgate, Shani Sniedze-Gregory, Caithlin Power, G. Barclay, D. Leech","doi":"10.37517/978-1-74286-685-7-9","DOIUrl":"https://doi.org/10.37517/978-1-74286-685-7-9","url":null,"abstract":"ACER works with thousands of Australian schools to set up Progressive Achievement Test (PAT) functionality, assign and administer tests, and to assist teachers to understand and interrogate the resulting test data and act upon findings. This session provides participants with an opportunity to hear about a school’s journey, and discuss this with the ACER team and our PAT school guest. ACER researchers have evaluated and documented assessment implementation and professional development opportunities, observing this school’s ability to track student and educator progress over time, which is a critical element in ACER’s Progressive Achievement approach. ACER’s educational consultants will demonstrate how PAT data ‘champions’ support staff across a primary and secondary school. Finally, we address and discuss the question many schools ask when using their data – ‘What does a year’s growth look like?’","PeriodicalId":191950,"journal":{"name":"Research Conference 2022: Reimagining assessment: Proceedings and program","volume":"22 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"131246596","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"ACER models of writing: Changing the assessment mindset","authors":"Juliette F Mendelovits, J. Nixon","doi":"10.37517/978-1-74286-685-7-4","DOIUrl":"https://doi.org/10.37517/978-1-74286-685-7-4","url":null,"abstract":"This presentation outlines different models of writing assessment that have a proven track record in various programs delivered by ACER. These models include holistic scoring, and partial analytical scoring on a number of criteria. The writing prompts for these models assume extended pieces of writing. The presentation will also introduce a new and alternative model of writing assessment that ACER has implemented in several large-scale assessments. This model is built on shorter pieces of writing from each student, designed to give more accurate and diverse insights into students’ proficiency in writing in a range of text types, within the constraints of limited assessment time. Each short piece is coded using a different marking scheme with customised criteria. The three models – holistic scoring, partial analytical scoring and customised criterion scoring – are designed for different assessment contexts, from high-stakes selection to formative use by classroom teachers.","PeriodicalId":191950,"journal":{"name":"Research Conference 2022: Reimagining assessment: Proceedings and program","volume":"57 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"133652903","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Reimagining the purpose of assessment","authors":"Geoffrey N. Masters","doi":"10.37517/978-1-74286-685-7-10","DOIUrl":"https://doi.org/10.37517/978-1-74286-685-7-10","url":null,"abstract":"Assessment is commonly understood as the process of judging how well students have learnt what they have been taught. It comes at the end of a sequence that begins with a curriculum or course syllabus. Teachers are expected to deliver this body of specified content, students are expected to learn it, and assessment is the process of judging and grading students on how well they have learnt what teachers have taught. This is a common view of assessment among students, parents and many teachers. I will argue in this presentation that this traditional understanding of what it means to learn successfully is no longer serving us well. It results in many less advanced students falling further behind in their learning, being written off as poor learners, and eventually disengaging from school. It also fails to challenge and extend some of our most advanced students, resulting in less progress than they are capable of making. The alternative is to use assessment to monitor learning – that is, to establish and understand where learners are in their long-term learning progress; to evaluate growth over time; and to provide teachers, students, parents, school leaders, system managers and governments with quality information to promote further learning. Shifting the focus from judging and grading to monitoring would be a significant change in assessment practice with the potential to better support successful learning.","PeriodicalId":191950,"journal":{"name":"Research Conference 2022: Reimagining assessment: Proceedings and program","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"121028901","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Assessment moderation: Is it fit for purpose?","authors":"L. Adie","doi":"10.37517/978-1-74286-685-7-2","DOIUrl":"https://doi.org/10.37517/978-1-74286-685-7-2","url":null,"abstract":"This presentation outlines the different practices and purposes of assessment moderation and analyses the benefits and issues of these. Two projects are presented; each seeks to improve teacher judgement. Both attempt to redefine what we mean by moderation and how moderation can be conducted to reach diverse and dispersed groups. The first draws on a current Australian Research Council project that is exploring the development of scaled exemplars, the use of commentaries of judgement decisions, and the use of digital platforms to support teachers grading student work. The second draws on the work of the Graduate Teacher Performance Assessment (GTPA) group of 19 universities to demonstrate how large-scale moderation of complex performance assessments can be conducted across states and territories. The layers of processes, including the use of technology, required to ensure judgement reliability are illustrated. In both contexts, the ways in which moderation is understood and practised are being reimagined.","PeriodicalId":191950,"journal":{"name":"Research Conference 2022: Reimagining assessment: Proceedings and program","volume":"23 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127216561","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}

{"title":"Reimagining classroom assessment and feedback to meet learner needs","authors":"F. M. Van der Kleij","doi":"10.37517/978-1-74286-685-7-5","DOIUrl":"https://doi.org/10.37517/978-1-74286-685-7-5","url":null,"abstract":"The power of classroom assessment and feedback to improve student learning outcomes has long been recognised. Yet, decades of research have yielded disappointing and often conflicting outcomes. This presentation challenges traditional conceptions of classroom assessment and feedback as teacher-driven practices. To meet learner needs better, it proposes a student-centred perspective in which students are active and have agency. By drawing on an extensive study of feedback reviews, this presentation illustrates how conceptualisations of feedback have changed over recent decades. This paper provides key insights into how classroom assessment and feedback practices can be changed in ways that are sustainable, afford student agency, and enhance student learning outcomes. Key learnings from recent research in primary and secondary education contexts are synthesised to provide state-of-the-art insights for shaping policy, practice and future research.","PeriodicalId":191950,"journal":{"name":"Research Conference 2022: Reimagining assessment: Proceedings and program","volume":"78 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2022-08-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"132535403","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}