Research analytics capabilities (RAC) survey: development, validation and revision using the Rasch model

Abstract

Purpose

The research enterprise within higher education is becoming more competitive as funding agencies require more collaborative research projects, higher-level of accountability and competition for limited resources. As a result, research analytics has emerged as a field, like many other areas within higher education to act as a data-informed unit to better understand how research institutions can effectively grow their research strategy. This is a new and emerging field within higher education.

Design/methodology/approach

As businesses and other industries are embracing recent advances in data technologies such as cloud computing and big data analytic tools to inform decision making, research administration in higher education is seeing a potential in incorporating advanced data analytics to improve day-to-day operations and strategic advancement in institutional research. This paper documents the development of a survey measuring research administrators’ perspectives on how higher education and other research institutions perceive the use of data and analytics within the research administration functions. The survey development process started with composing a literature review on recent developments in data analytics within the research administration in the higher education domain, from which major components of data analytics in research administration were conceptualized and identified. This was followed by an item matrix mapping the evidence from literature with corresponding, newly drafted survey items. After revising the initial survey based on suggestions from a panel of subject matter experts to review, a pilot study was conducted using the revised survey instrument and validated by employing the Rasch measurement analysis.

Findings

After revising the survey based on suggestions from the subject matter experts, a pilot study was conducted using the revised survey instrument. The resultant survey instrument consists of six dimensions and 36 survey items with an establishment of reasonable item fit, item separation and reliability. This survey protocol is useful for higher educational institutions to gauge research administrators’ perceptions of the culture of data analytics use in the workplace. Suggestions for future revisions and potential use of the survey were made.

Originality/value

Very limited scholarly work has been published on this topic. The use of data-informed and data-driven approaches with in research strategy within higher education is an emerging field of study and practice.