***************************************************************** ** Second Workshop on Empirical Evaluation of Adaptive Systems ** ** held in conjunction with UM2003, Pittsburgh ** ** http://art.ph-freiburg.de/um2003 ** ***************************************************************** You are kindly invited to contribute to the Second Workshop on Empirical Evaluation of Adaptive Systems, which follows in the tracks of a very successful first workshop held in conjunction with UM2001. This workshop intends to explore methods and criteria that can be employed in assessing (interactive) adaptive systems. The perspective taken is that a build-up of knowledge in this direction can serve as a driving force in facilitating and fostering detailed empirical studies with significant results in the field of user modeling  something lacking from the literature today. The workshop seeks to cover a wide range of application domains and user modeling techniques and aims to both raise awareness of researchers and practitioners in the area, as well as contribute towards a widely accepted evaluation "toolset" for adaptive systems. Goals and Workshop Format The workshop is organized around three central questions: - What criteria and empirical methods can be used for the evaluation of adaptive systems? What new criteria need to be introduced to specifically cater for the presence of adaptation in the evaluated systems? What empirical methods are appropriate (or, how do existing methods need to be modified, so as to be suitable for) assessment against the new sets of criteria? - Can metrics be developed to facilitate the comparison between different (versions of) adaptive systems, or between adaptive and non-adaptive systems? - How can we foster an increase in the volume and quality of empirical evaluations of adaptive systems? What are the most common pitfalls that can be identified in previous studies? Each submission is required to address at least one of these questions. The goals of this workshop are, on the one hand, to consolidate existing knowledge in the area of adaptive system evaluation, and, on the other hand, to provide researchers and practitioners in the field with a set of "tools" and guidelines, which could foster future work and facilitate a more homogeneous view of related evaluation activities. Participants of the workshop are expected to share their experiences and discuss the issue of what constitutes good and bad practice in studies targeting adaptive systems. Two types of submissions will be solicited to achieve this goal: studies and position-papers. Studies Submitted studies should contain a detailed description of the empirical design, the criteria, and results that were obtained when evaluating one (or more) adaptive systems. Each study will be discussed at the workshop in terms of experience gathered. The submission of "unsuccessful" studies is explicitly encouraged, as these can be very informative for other participants. Furthermore, we would like to encourage the submission of possibly preliminary studies, which follow innovative approaches to the assessment of adaptation, or have arrived at results that may be of particular interest to workshop participants. Position-papers Position-papers should also address one or more of the above questions. Controversial positions that may stimulate a fertile discussion at the workshop will be preferred. During the workshop we will work towards an enumeration of factors and practices that are known to have beneficial, or, conversely, detrimental effects on the design and performance of evaluation studies on adaptive systems. Moreover, authors of studies will be asked to categorize their evaluation against a framework introduced by the workshop organizers. This framework (which is a merger of two frameworks that have proposed previously) will be used to "tie together" and amalgamate the results of the workshop. We intend to consolidate the framework by relating the submitted studies to it and by discussing its usefulness and feasibility. Submission Format As stated above, each submission is required to address at least one of the main workshop questions. Fulfillment of this requirement will be assessed in the course of the review process. Submissions should not exceed 10 pages and should be formatted according to the LLNCS style. Please refer to the workshop's web site for more information on the submission format: http://art.ph-freiburg.de/um2003. Workshop papers will be published in full length in the workshop proceedings and presented in talks at the workshop. Note that workshop participants are not (!) required to register for the whole conference, but may register for the workshop only (http://www2.sis.pitt.edu/~um2003). Please, submit your paper by e-mail to Stephan Weibelzahl (weibelza@ph-freiburg.de) until March 3, 2003. Postscript (*.ps) and Portable Document Format (*.pdf) files are preferred. Important dates March 3, 2003: submission of papers and position papers March 31, 2003: notification of authors April 28, 2003: delivery of camera ready copy June 22, 2003: workshop date; the main conference will last until June 26, 2003 Any questions? Don't hesitate to ask the organizers Stephan Weibelzahl weibelza@ph-freiburg.de University of Education Freiburg, Germany Alexandros Paramythis alpar@fim.uni-linz.ac.at Johannes Kepler University Linz, Austria