Curriculum analytics adoption in higher education: A multiple case study engaging stakeholders in different phases of design

Abstract

Several studies have indicated that stakeholder engagement could ensure the successful adoption of learning analytics (LA). Considering that researchers and tech developers may not be aware of how LA tools can derive meaningful and actionable information for everyday use, these studies suggest that participatory approaches based on human-centred design can provide stakeholders with the opportunity to influence decision-making during tool development. So far, there is a growing consensus about the importance of identifying stakeholders' needs and expectations in early stages, so researchers and developers can design systems that resonate with their users. However, human-centred LA is a growing sub-field, so further empirical work is needed to understand how stakeholders can contribute effectively to the design process and the adoption strategy of analytical tools. To illustrate mechanisms to engage various stakeholders throughout different phases of a design process, this paper presents a multiple case study conducted in different Latin American universities. A series of studies inform the development of an analytical tool to support continuous curriculum improvement, aiming to improve student learning and programme quality. Yet, these studies differ in scope and design stage, so they use different mechanisms to engage students, course instructors and institutional administrators. By cross analysing the findings of these three cases, three conclusions emerged for each design phase of a CA tool, presenting mechanisms to ensure stakeholder adoption after tool development. Further implications of this multiple case study are discussed from a theoretical and methodological perspective.

Practitioner notes

What is already known about this topic?

• Human-centred learning analytics (LA) has accommodated different configurations of stakeholder engagement, including co-design and participatory design.
• Participatory design provides developers with a wide variety of techniques to engage a particular group in a mutual learning process.
• Most studies mainly focus on engaging stakeholders to identify needs in the early stages of the design process.
• More empirical works are needed to unveil the effectiveness of human centredness during LA design and after tool development.

What this paper adds?

• Provides a multiple case study to illustrate mechanisms to engage various stakeholders in different design phases of a curriculum analytics (CA) tool.
• Summarises different assertions based on case study findings regarding needs for analytical tool, its early evaluation and its potential use after development.
• Provides empirical evidence on how to promote stakeholder engagement at a specific design stage and for a specific purpose.

The implications for practitioners

• LA researchers and developers can use assertions based on evidence as a starting point to drive the design process of CA solutions.
• Researchers and practitioners will have a set of protocols to implement participatory techniques in different design phases of a CA tool.