The Development of a Literacy-Based Research Integrity Assessment Framework for Graduate Students in Taiwan.

Abstract

Graduate education is a critical period in shaping and fostering graduate students' awareness about the importance of responsible conduct of research and knowledge and skills in doing good science. However, there is a lack of a standard curriculum and assessment framework for graduate students in Taiwan. The aim of this study was to develop a literacy-based research integrity (RI) assessment framework, including five core RI areas: (1) basic concepts in RI, (2) RI considerations in the research procedure, (3) research ethics and research subject protection, (4) publication and authorship, and (5) conflict of interest. The five areas were derived through a comprehensive review of major topics and areas covered in existing research integrity education and training programs and were rated by RI experts with adequate content validity. Test items on the five core areas were developed across three literacy levels: remembering and understanding, applying and analyzing, and evaluating and creating. Seven thousand and eighty-seven graduate-level trainees took an 18-unit RI course covering the five RI areas. Upon finishing the course, trainees completed a computer-based RI assessment randomly selected from 26 RI testing booklets. The design of test items followed the mastery-oriented assessment principles to promote trainees' learning of RI with adaptive assessment feedback. Results showed that the items in the RI assessment had adequate discrimination and low difficulty level. Thus, the RI assessment can be used to assess a range of trainees' RI literacy and can provide the most information in identifying trainees in need of more instruction or alternative training. The low guessing parameters also indicated the online RI assessment had an appropriate control of test exposure and cheating prevention. Higher education authorities can use this framework to assess graduate students' RI literacy based on a standard curriculum and prepare them for conversations about the responsible conduct of research for RI culture-building.