*****************************************************
                  Call for Papers
 Workshop on Empirical Evaluations of Adaptive Systems
            in conjunction with UM2001
                  July 13, 2001
                Sonthofen, Germany
 *****************************************************

  Empirical evaluations of adaptive systems are
  hard to find, e.g., only a quarter of the
  articles published in UMUAI are reporting
  significant empirical evaluations. Several
  reasons have been identified to be responsible
  for this lack. These include both structural as
  well as adaptivity specific reasons. However,
  user modeling systems are designed for
  human-computer interaction and thus, an
  empirical approach is absolutely necessary if we
  want to justify the enormous efforts which are
  needed to make systems adaptive. The aim of this
  full-day workshop is to systematize, coordinate
  and encourage research on evaluation issues by
  bringing together people who are interested in
  comparing research in different domains,
  identifying common problems, and discussing
  possible solutions.

Workshop Format 

  Each session focuses on a few exemplary
  evaluations in a specific domain: participants
  are required to bring along a description of a
  finished or planned empirical evaluation. Each
  evaluation will be introduced by a talk of 5-10
  minutes (depending on the total amount of
  participants). Evaluations that have been
  completed will be discussed in terms of what
  worked well, what went wrong, and how it might
  be done better next time, etc. Discussions on
  planned evaluations will identify strengths,
  weaknesses, pitfalls, and possible methods of
  analysis based on the experience of the group
  members with similar cases. This workshop format
  will help encourage discussion, since the
  participants will be discussing specific
  empirical evaluations that they have done or
  plan to do rather than evaluations in the
  abstract.

The topics of interest include, but are not
limited to, the following list:
 - evaluation frameworks for user-modeling systems
 - introduction, comparison or, discussion of
   evaluation criteria
 - influence of adaptations on behavioral or
   cognitive variables
 - evaluation of the accuracy or correctness of
   either user models or inference techniques
 - effectiveness of user adaptations for task
   accuracy, task completion time, task quality
 - evaluation methods and techniques (experimental
   design vs. field studies; transfer of usability
   techniques; questionnaire design; behavior
   observation; eye-tracking; etc.)
 - qualitative methods applied to user modeling
 - design-guidelines or heuristics inferred from
   empirical evaluations
 - solutions of problems that are specific for the
   evaluation of adaptivity
 - investigation of evaluation strategies

Submission and Information 

  Due to the workshop format there are several
  submission requirements. Please, refer to the
  workshop web page for detailed information and
  submission instructions:
  http://art2.ph-freiburg.de/um2001/index.html

Important Dates 

 March 8, 2001: submission deadline for workshop papers
 April 1, 2001: notification of authors
 July 13, 2001: workshop


Workshop Organizers 

 Stephan Weibelzahl
  PH Freiburg, Germany
  weibelza@uni-freiburg.de 

 David Chin
  University of Hawaii, USA
  chin@hawaii.edu 

 Gerhard Weber
  PH Freiburg, Germany
  webergeh@ph-freiburg.de