Problems of analysis of the relationship between the learning context and TIMSS testing results

Abstract

Introduction. For more than two decades, the Organisation for Economic Cooperation and Development has been organising a number of comparative studies of the quality of education in different countries. One of them is the study of mathematics and science education TIMSS (Trends in International Mathematics and Science Study), which is conducted jointly with the International Association for the Evaluation of Educational Achievements (IEA). The last, seventh, cycle of the study was conducted in 2019. TIMSS statistics is regularly posted in the public domain on the IEA website so that specialists can independently conduct research in any aspect of their interest. One of the areas of analysis in this case is traditionally the search for the causes of certain test results, which are determined by the peculiarities of the organisation of the educational process and the context of learning in different countries. At the same time, a study of professional literature showed that among the factors of the social and school context, the analysis of which is provided by the research tools, only an extremely limited range of them turned out to be statistically directly related to TIMSS scores. Specialists systematically encounter inexplicable absence or low correlation values of TIMSS test scores and context indicators. The authors think that the main reason for such difficulties is inattention to the peculiarities of the indicators used in the calculation of measures of statistical relationships. Aim. The present research aims to identify a statistical relationship between test results and indicators of the context of schoolchildren’s learning, as well as the influence of the TIMSS information collection and processing system on the productivity of analysing the research results.Methodology and research methodology. The methodological basis of the work is a systematic approach, which is based on the consideration of the results of the international TIMSS study as a whole: i.e. a complex of interrelated elements (organisations, tools, assessment indicators, scoring systems). The work was carried out on the basis of applied research procedures (observation, description, comparison, measurement, etc.), within which general scientific (comparative analysis, systematisation, generalisation) and statistical research methods (statistical and correlation analysis, etc.) were also used. The source of information was the International Database of Electronic Testing TIMSS-2019, hosted in the IEA repository. The TIMSS datasets were analysed using the IEA International Database (IDB) parser plug-in for SPSS (version 4.0).Results. For most indicators of the social and student context of learning, the authors found the absence or low value of statistical relationships with TIMSS scores. The number of books at home and parents’ education turned out to be statistically related to TIMSS scores concerning the indicators of social well-being and home learning conditions envisaged by the organisers. The indicators of learning conditions at school included the frequency of independent work in class; motivational factors included plans to continue education and self-evaluation of students’ math proficiency. Evidently, even these relationships turned out to be weak. It was revealed that the difficulties in detecting a correlation between TIMSS scores and learning conditions are caused by the very nature of the analysed variables: 1. the approximate nature of individual student assessments used in TIMSS; 2. low differentiation of students according to a number of indicators of the learning context; 3. insufficient reliability of information obtained from sociological surveys of schoolchildren.Practical significance. The authors believe that in order to improve the quality of analytical work on relevant topics, it is necessary to pay close attention to the essence behind the variables used in statistical calculations. In turn, the TIMSS organisers need to continue improving the measurement procedures and research tools by introducing additional success criteria that reflect the individual and comparable results of students in the current TIMSS cycle, as well as indicators of the reliability of contextual information obtained by sociological means.