 | Literature Review on the Evaluation of
Adaptive SystemsOn this page you can download a database
containing reports of user-centered evaluations of adaptive and /
or adaptable systems. It is related to an article by Van Velsen et.
al, in which the user-centered evaluation practice of
personalized systems was reviewed. The database includes short
descriptions of each evaluation, including, among others, the
methods used, variables assessed and lessons learned about how to
evaluate a personalized system. Interested researchers are hereby
granted permission to use the data (without changing it any way)
for private research purposes. If you wish to use this data in a
published article, you will need to ask for persmission from the
paper's first author.
Van Velsen, L., Van der Geest, T., Klaassen, R.
& Steehouder, M. (2008).
User-centered evaluation of adaptive and
adaptable systems: a literature review..The
Knowledge Engineering Review, 23(3), 261-281.
doi:10.1017/S0269888908001379 This literature review focuses on
user-centered evaluation (UCE) studies of adaptive and adaptable
systems. Usability, perceived usefulness and appropriateness of
adaptation are the three most commonly assessed variables.
Questionnaires appeared to be the most popular method, followed
by interviews and data log analysis. The quality of most
questionnaires was questionable, and the reporting of interviews
and think-aloud protocols was found to be shallow. Furthermore,
data logs need triangulation in order to be useful. The reports
encountered lack empirical value. The article models the
iterative design process for adaptive and adaptable systems,
linked to the goals of UCE: supporting decisions, detecting
problems and verifying quality. This model summarizes the
variables to be assessed at each stage of the process and the
relevant methods to assess them.
|
