Developing and validating a rubric-based approach to quality assurance in Turkish higher education

IF 2 4区社会学 Q2 SOCIAL SCIENCES, INTERDISCIPLINARY

Evaluation and Program Planning Pub Date : 2025-08-09 DOI:10.1016/j.evalprogplan.2025.102673

Buket Akkoyunlu , Salih Bardakcı , Sibel Aksu Yıldırım , M. Dilek Avşaroğlu , Gonca Uludağ , Ayhan Koçer , Muzaffer Elmas

{"title":"Developing and validating a rubric-based approach to quality assurance in Turkish higher education","authors":"Buket Akkoyunlu , Salih Bardakcı , Sibel Aksu Yıldırım , M. Dilek Avşaroğlu , Gonca Uludağ , Ayhan Koçer , Muzaffer Elmas","doi":"10.1016/j.evalprogplan.2025.102673","DOIUrl":null,"url":null,"abstract":"<div><div>This study, initiated by the Turkish Higher Education Quality Council (THEQC), develops and validates a rubric-based approach for evaluating institutional quality assurance (QA) processes within the Turkish higher education ecosystem. Effective quality assurance (QA) in higher education requires evaluation tools that are methodologically robust, responsive to institutional needs, and aligned with international benchmarks. A rubric-based approach offers a structured and transparent framework for consistent assessments. The rubric was created through an iterative, evidence-based process incorporating expert feedback, international benchmarks, site visits, and prior research. It consists of 22 main criteria across four key domains—Leadership, Governance and Quality; Learning and Teaching; Research and Development; and Service to Society—subdivided into 46 sub-criteria, each assessed on a five-point maturity scale. The validity and reliability of the rubric were rigorously examined. Content validity was established through evaluations by 10 QA professionals, addressing four key dimensions: relevance, alignment with the intended domain, appropriateness for the target audience, and clarity of language. Construct and criterion validity were assessed through consistency checks and usability analyses, with data collected from 252 external evaluators across 57 higher education institutions. Additionally, 360-degree feedback was gathered from both external evaluation teams and the institutions involved in the evaluation process.</div></div><div><h3>Results</h3><div>demonstrate strong content validity (Lawshe coefficient = 1.00, p < .05), high interrater consistency, and strong usability across various evaluator groups. Further evaluation confirmed the rubric’s applicability and robustness across diverse institutional contexts. This study concludes that the developed rubric is a reliable and valid tool for enhancing and evaluating quality assurance practices in higher education.</div></div>","PeriodicalId":48046,"journal":{"name":"Evaluation and Program Planning","volume":"113 ","pages":"Article 102673"},"PeriodicalIF":2.0000,"publicationDate":"2025-08-09","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Evaluation and Program Planning","FirstCategoryId":"90","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S0149718925001405","RegionNum":4,"RegionCategory":"社会学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"SOCIAL SCIENCES, INTERDISCIPLINARY","Score":null,"Total":0}

引用次数: 0

Abstract

This study, initiated by the Turkish Higher Education Quality Council (THEQC), develops and validates a rubric-based approach for evaluating institutional quality assurance (QA) processes within the Turkish higher education ecosystem. Effective quality assurance (QA) in higher education requires evaluation tools that are methodologically robust, responsive to institutional needs, and aligned with international benchmarks. A rubric-based approach offers a structured and transparent framework for consistent assessments. The rubric was created through an iterative, evidence-based process incorporating expert feedback, international benchmarks, site visits, and prior research. It consists of 22 main criteria across four key domains—Leadership, Governance and Quality; Learning and Teaching; Research and Development; and Service to Society—subdivided into 46 sub-criteria, each assessed on a five-point maturity scale. The validity and reliability of the rubric were rigorously examined. Content validity was established through evaluations by 10 QA professionals, addressing four key dimensions: relevance, alignment with the intended domain, appropriateness for the target audience, and clarity of language. Construct and criterion validity were assessed through consistency checks and usability analyses, with data collected from 252 external evaluators across 57 higher education institutions. Additionally, 360-degree feedback was gathered from both external evaluation teams and the institutions involved in the evaluation process.

Results

demonstrate strong content validity (Lawshe coefficient = 1.00, p < .05), high interrater consistency, and strong usability across various evaluator groups. Further evaluation confirmed the rubric’s applicability and robustness across diverse institutional contexts. This study concludes that the developed rubric is a reliable and valid tool for enhancing and evaluating quality assurance practices in higher education.

查看原文本刊更多论文

开发和验证基于规则的方法来保证土耳其高等教育的质量

本研究由土耳其高等教育质量委员会（THEQC）发起，开发并验证了一种基于规则的方法，用于评估土耳其高等教育生态系统中的机构质量保证（QA）过程。在高等教育中，有效的质量保证（QA）需要评估工具，这些工具在方法上是稳健的，对机构的需求做出反应，并与国际基准保持一致。基于规则的方法为一致的评估提供了一个结构化和透明的框架。该准则是通过一个反复的、以证据为基础的过程创建的，该过程结合了专家反馈、国际基准、实地考察和先前的研究。它包括22项主要标准，涉及四个关键领域：领导力、治理和质量；学与教；研究与开发；和社会服务——细分为46个子标准，每个标准以五分制成熟度量表进行评估。对该指标的有效性和可靠性进行了严格的检验。内容有效性是通过10名QA专业人员的评估来建立的，涉及四个关键维度：相关性、与预期领域的一致性、对目标受众的适当性以及语言的清晰度。通过一致性检查和可用性分析来评估结构和标准的效度，数据来自57所高等教育机构的252名外部评估者。此外，从外部评估小组和参与评估过程的机构收集了360度的反馈。结果显示，在不同的评估者群体中，具有较强的内容效度（Lawshe系数= 1.00,p <； .05）、较高的评估者一致性和较强的可用性。进一步的评估证实了该准则在不同制度背景下的适用性和稳健性。本研究的结论是，所开发的指标是一个可靠和有效的工具，以加强和评估高等教育的质量保证实践。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文求助全文

来源期刊

Evaluation and Program Planning SOCIAL SCIENCES, INTERDISCIPLINARY-

CiteScore

3.10

自引率

6.20%

发文量

112

期刊介绍： Evaluation and Program Planning is based on the principle that the techniques and methods of evaluation and planning transcend the boundaries of specific fields and that relevant contributions to these areas come from people representing many different positions, intellectual traditions, and interests. In order to further the development of evaluation and planning, we publish articles from the private and public sectors in a wide range of areas: organizational development and behavior, training, planning, human resource development, health and mental, social services, mental retardation, corrections, substance abuse, and education.