 | Assessment ToolsThese are the first entries of what we hope will become a comprehensive collection of assessment tools, questionnaires and methods that can be used for the evaluation of adaptive systems. We would like to encourage other researchers to open up and share their evaluation material here. The main advantage will be, that evaluation results become more comparable. While currently each study applies its own criteria, if several studies were to, for instance, use the same questionnaire across different systems, this would potentially improve the interpretation and validation of the acquired results. Furthermore, this site could just simplify the evaluation process, as researchers would not need to "reinvent wheels" by developing assessment materials from scratch. Note: In order to "protect" your material, you might want to distribute it under a free license such as GNU GPL (for software), GNU FDL (for documentation) or Creative Commons (for other material). Quick index: Questionnaire for adaptive learning courses. Comment This questionnaire (in German) has been used for the evaluation of several adaptive learning courses which are based on the authoring system NetCoach (e.g., the HTML-Tutor). The assessed data includes the learners' subjective impression of the course in general, as well as their attitude towards the adaptive features in particular. Although the questionnaire has not detected many differences between various adaptation variants (adaptive curriculum sequencing and guiding) so far, it may serve as a useful starting point for derivative work. Collection of usability questionnaires. Comment Usability of systems is often assessed with questionnaires. Gary Perlman provides a nice overview and examples of existing questionnaires. However, please note that these questionnaires are not adaptation-specific and might under certain circumstances even be inappropriate for the evaluation of an adaptive system. Analysing web browsing behaviour. Comment If your empirical study includes the analysis of web browsing behaviour, you might be interested in using SCONE, "which was designed to allow the quick development and evaluation of new Web enhancements for research and educational purposes". Testbed for metrics of website usage. Comment The National Institute of Standards and Technology (NIST) provides a so called Web Metrics Testbed. It contains a set of tools to analyse the usability of websites. |