Fitting Free-Form Question-Asking and Spatial Ability into ITS Development
Thesis DisciplineComputer Science
Degree GrantorUniversity of Canterbury
Degree NameMaster of Science
Intelligent Tutoring Systems (ITSs) are problem-solving environments that provide individualised instruction and are able to adapt to the abilities and needs of each individual student in order to maximise effective learning. They provide feedback on students' actions, but a problem arises when students do not always understand the feedback they receive. Therefore, it would be beneficial for students to be able to ask for additional clarifications at any time, and to receive feedback customised to their individual differences. This research focuses on providing an additional help channel in ITSs where students are able to ask free-form questions, as well as accounting for the students' psychometric measure of spatial ability. We describe ERM-Tutor, the test-bed ITS chosen for implementing our research framework. ERM-Tutor is a constraint-based tutoring system for teaching logical database design. Students practise this procedural task in ERM-Tutor by solving each step and receiving feedback on their solutions. We also present our approach to addressing the meta-cognitive skill of question-asking in ERM-Tutor. We added a question-asking module that enables students to ask free-form questions and receive the most appropriate answers stored in the system. In addition, we investigated the potential of tailoring the feedback messages towards the learners' psychometric measure of spatial ability. We modified ERM-Tutor to provide not only textual feedback messages, but also multimedia messages, containing a combination of text and pictures. We performed a series of evaluation studies in order to evaluate the effectiveness of our proposed solutions. All our studies were conducted with tertiary students enrolled in an introductory database course. The students had attended lectures on logical database design and were asked to use ERM-Tutor to develop and practise their mapping skills. The results show an overall improvement in performance and learning gain for all students using ERM-Tutor. Interactions with the question-asking module show that most questions asked by students were task-focused, directly requesting help on specific errors. The results confirm the need for addressing students' questions inside an ITS environment. Furthermore, there were no conclusive results to support a difference in effectiveness of the textual versus multimedia feedback presentation modes with respect to the students' spatial ability. However, we observed a number of trends indicating that matching the instruction presentation mode towards the students spatial ability influences their perception of the system and motivation to use it, more than their learning gain. Our results show promising indications for further explorations. We present our approaches, full analyses of the collected data from our evaluation studies, as well as our research contributions to the ITSs field. We also portray a number of future directions that will contribute towards maximising the effectiveness of learning in ITSs.