Lessons learned from designing a curriculum analytics tool for improving student learning and program quality

Abstract

Curriculum Analytics (CA) emerged as a sub-field of Learning Analytics, aiming to use large amounts of educational data to drive curriculum decision-making and program improvement. However, it is still an open question how the use of CA tools impacts student learning and program quality. To advance this field, this paper describes the lessons learned from having designed and implemented a CA tool to help managers and teaching staff reflect on curriculum and teaching practices. This CA tool was developed under a design-based research approach called The Integrative Learning Design Framework. We implemented a two-cycle building-testing structure to evaluate the perceived usefulness and usability of this tool. The first cycle consisted of designing a first version of the tool and evaluating its use throughout a case study involving 5 managers and 124 teaching staff members who participated in a 3-year continuous improvement process in one Latin American university. The second cycle consisted of redesigning the tool according to the lessons learned during the first cycle and evaluating its use throughout workshops with 16 managers and 9 teaching staff members in two Latin American universities. Findings indicate that the CA tool helped teachers collect a greater number and variety of evidence regarding students’ attainment of competencies, allowing staff to be more aware of the learning situation of their students when redesigning course assessment methods and course sequences. Currently, this CA tool is being used by 20 Latin American universities, guiding curriculum renewal strategies beyond the current global pandemic.