Evaluating and improving adaptive educational systems with learning curves

Type of content
Journal Article
Thesis discipline
Degree name
Publisher
University of Canterbury. Computer Science and Software Engineering
Journal Title
Journal ISSN
Volume Title
Language
Date
2011
Authors
Martin, B.
Mitrovic, Antonija
Koedinger, K.
Mathan, S.
Abstract

Personalised environments such as adaptive educational systems can be evaluated and compared using performance curves. Such summative studies are useful for determining whether or not new modifications enhance or degrade performance. Performance curves also have the potential to be utilised in formative studies that can shape adaptive model design at a much finer level of granularity. We describe the use of learning curves for evaluating personalised educational systems and outline some of the potential pitfalls and how they may be overcome. We then describe three studies in which we demonstrate how learning curves can be used to drive changes in the user model. First, we show how using learning curves for subsets of the domain model can yield insight into the appropriateness of the model’s structure. In the second study we use this method to experiment with model granularity. Finally, we use learning curves to analyse a large volume of user data to explore the feasibility of using them as a reliable method for fine-tuning a system’s model. The results of these experiments demonstrate the successful use of performance curves in formative studies of adaptive educational systems.

Description
Citation
Martin, B., Mitrovic, A., Koedinger, K., Mathan, S. (2011) Evaluating and improving adaptive educational systems with learning curves. User Modeling and User-Adapted Interaction, 21(3), pp. 249-283.
Keywords
empirical evaluation, intelligent tutoring systems, student modeling, user modelling, domain modelling, learning curves
Ngā upoko tukutuku/Māori subject headings
ANZSRC fields of research
Fields of Research::39 - Education::3904 - Specialist studies in education::390402 - Education assessment and evaluation
Rights