The validity and inter-rater reliability of project assessment in mathematics learning

Abstract

[English]: A common criticism of project assessment is the subjectivity and inconsistency of raters in scoring. In the present article, we provide the result of validity and inter-rater reliability test of the project assessment instrument. The instrument with a rubric was used to assess students’ project task in grade eight for function and relation topic. The task was adopted from mathematics textbooks used in the schools. The instrument has been tested to 10 raters/teachers and 94 grade eight students from three schools (in Surabaya and Gresik). Data were collected through the project assessment sheet along with its rubric as the scoring guidance for the teachers. Construct validity was analyzed through confirmatory factor analysis, while a reliability test was conducted by using inter-rater reliability method with the Intraclass Correlation Coefficient. The result of the validity test showed that the instrument did not fulfill the criteria of construct validity. It is indicated by the different number of factors between the initial construction and the empirical test result. In term of inter-rater reliability, the instrument is highly reliable. The findings indicate the need for testing the non-test assessment instrument provided on mathematics textbooks, so the aspects of its assessment sheet fulfill the valid and reliable criteria. Keywords: Validity, Reliability, Inter-rater, Project assessment, Project task [Bahasa]: Subjektivitas dan kurang konsistennya guru/rater dalam proses penyekoran merupakan kritik yang umum ditujukan pada penilaian proyek dalam pembelajaran matematika. Oleh karena itu, artikel ini menyajikan hasil uji validitas konstruk dan reliabilitas inter-rater instrumen penilaian projek. Instrumen yang dilengkapi rubrik tersebut digunakan untuk menilai tugas proyek siswa kelas VIII SMP pada materi relasi dan fungsi. Tugas diambil dari buku matematika yang digunakan di sekolah. Instrumen diujicobakan pada 10 raters/guru dan 94 siswa di tiga sekolah berbeda di Kota Surabaya dan Kabupaten Gresik. Data dikumpulkan melalui lembar penilaian projek yang dilengkapi rubrik penilaian sebagai pedoman guru dalam melakukan penskoran. Validitas konstruk dianalisis dengan menggunakan Confirmatory Factor Analysis, sedangkan reliabilitas inter-rater dianalisis dengan Intraclass Correlation Coefficient. Hasil uji validitas menunjukkan bahwa instrumen penilaian yang digunakan tidak valid secara konstruk. Ketidakvalidan ditandai dengan perbedaan banyaknya faktor hasil konstruksi awal dengan hasil uji empiris. Dari sisi reliabilitas inter-rater, instrumen penilaian proyek yang digunakan reliabel. Temuan ini mengindikasikan perlunya dilakukan pengujian instrumen penilaian non tes pada buku matematika, sehingga aspek-aspek dalam lembar penilaian menjadi valid dan reliabel. Kata kunci: Validitas, Reliabilitas, Inter-rater, Penilaian proyek, Tugas proyek