UM 2007 Tutorial
"Evaluation 2: Formative Evaluation Methods for Adaptive Systems"

Description

Adaptive systems are by definition interactive systems. As such they stand to benefit considerably from a development lifecycle that ensures user involvement from the early design stages, and embraces evaluation, in both formative and summative forms.

The guiding perspective of this tutorial is that adequate evaluation methods, reliable assessment criteria and metrics are prerequisites for improving the quality and usability of the next generations of adaptive systems.

Upon successful completion of the tutorial, participants will

• Be aware of the specific problems involved in the evaluation of adaptive systems that differentiate them from their non-adaptive counterparts, and able to solve or circumvent these problems
• Understand and be able to apply the principles of layered evaluation of adaptive systems
• Be able to design a targeted formative evaluation study for an adaptive system (e.g., addressing a given layer and set of criteria) by selecting appropriate methods and criteria

Prerequisites

We recommend that participants who have no prior knowledge in designing empirical studies and statistical analysis visit the morning session "Evaluation 1: Fundamental Empirical Techniques and Caveats" before they attend this tutorial.

Alternatively, participants will be offered the opportunity to read a small set of preparatory materials available on-line before attending the tutorial.

Format

We aim to make the learning experience of this tutorial as interactive and learner-centred as possible. The sessions will use a combination of interactive lectures and practical application of methods to case studies. Participants will be split into groups (of 5-7 participants) to work on a case study of their choice.

Lecture notes will be made available during the tutorial and afterwards on the website.

Date and time

Tuesday, June 26th 2007, Afternoon session

Organizers

Dr. Stephan Weibelzahl
National College of Ireland Dublin
Mayor Street IFSC
Dublin 1
Ireland
+353 1 4498 579
sweibelzahl at ncirl.ie
http://www.weibelzahl.de/

Alexandros Paramythis
Institute for Information Processing and Microprocessor Technology (FIM)
Johannes Kepler University Linz
Altenbergerstr. 69
A-4040 Linz, AUSTRIA
+43 732 2468 8442
alpar at fim.uni-linz.ac.at
http://www.fim.uni-linz.ac.at/staff/paramythis/

Dr. Judith Masthoff
Department of Computing Science
University of Aberdeen
Aberdeen AB24 3UE
Scotland, UK
+44 1224 272299
jmasthoff at csd.abdn.ac.uk
http://www.csd.abdn.ac.uk/~jmasthof/