Call for Papers

Empirical evaluations of adaptive systems are hard to find, e.g., only a quarter of the articles published in UMUAI are reporting significant empirical evaluations. Several reasons have been identified to be responsible for this lack. These include both structural as well as adaptivity specific reasons. However, user modeling systems are designed for human-computer interaction and thus, an empirical approach is absolutely necessary if we want to justify the enormous efforts which are needed to make systems adaptive. The aim of this full-day workshop is to systematize, coordinate and encourage research on evaluation issues by bringing together people who are interested in comparing research in different domains, identifying common problems, and discussing possible solutions.

 Workshop Format

Each session focuses on a few exemplary evaluations in a specific domain: participants are required to bring along a description of a finished or planned empirical evaluation. Each evaluation will be introduced by a talk of 5-10 minutes (depending on the total amount of participants). Evaluations that have been completed will be discussed in terms of what worked well, what went wrong, and how it might be done better next time, etc. Discussions on planned evaluations will identify strengths, weaknesses, pitfalls, and possible methods of analysis based on the experience of the group members with similar cases. This workshop format will help encourage discussion, since the participants will be discussing specific empirical evaluations that they have done or plan to do rather than evaluations in the abstract.

The topics of interest include, but are not limited to, the following list:

• evaluation frameworks for user-modeling systems
• introduction, comparison or, discussion of evaluation criteria
• influence of adaptations on behavioral or cognitive variables
• evaluation of the accuracy or correctness of either user models or inference techniques
• effectiveness of user adaptations for task accuracy, task completion time, task quality
• evaluation methods and techniques (experimental design vs. field studies; transfer of usability techniques; questionnaire design; behavior observation; eye-tracking; etc.)
• qualitative methods applied to user modeling
• design-guidelines or heuristics inferred from empirical evaluations
• solutions of problems that are specific for the evaluation of adaptivity
• investigation of evaluation strategies

 Submission and Information

Due to the workshop format there are several submission requirements. Please, refer to the submission page for detailed information and submission instructions.

 Important Dates

• March 1, 2001: submission of abstract (recommended)
• March 15, 2001: extended submission deadline for workshop papers
• April 1, 2001: notification of authors
• July 13, 2001: workshop

 Workshop Organizers

• Stephan Weibelzahl
  PH Freiburg, Germany
  weibelza@ph-freiburg.de
• David Chin
  University of Hawaii, USA
  chin@hawaii.edu
• Gerhard Weber
  PH Freiburg, Germany
  webergeh@ph-freiburg.de

 This workshop is recommended by Ben Shneiderman!