Annual report on the PUSH project: September 1995 - May 1996.

(This report can also be found in http://www.sics.se/~kia/annual_report_nutek_push.html>)

The PUSH project started in September 1993, and aims to finish in June 1996. The two main goals of the project was to develop techniques for making systems adaptive, and to empirically establish their usefulness. We also aimed at finding good design principles for adaptive systems, and to integrate the adaptivity in a multi-modal setting. Previous work includes:

Studies of the system

During this last year of the PUSH project, we have completed several important goals of the project. Most important is perhaps, that we were able to turn our prototype adaptive hypermedia system, POP, into such a state that we could evaluate its usefulness with real users. We did this through two studies. The first, done in December 1995, was aimed at evaluating the usefulness of the interface and it was also part in bootstrapping the adaptivity of the system. The evaluation was done on a non-adaptive variant of the system. This evaluation has been reported in Espinoza and Höök, (1996b). Thirteen users participated in this study. They had some comments on the interface which we later on incorporated in a second version of the prototype evaluated in Febraury 1996.

The second study, in February 1996, was a comparative study, where we compared the adaptive systems with a non-adaptive variant of our system. The results from this study are not yet evaluated in full detail. What we know so far is that the users preferred the adaptive system over the non-adaptive variant. They found the adaptations to be good and relevant. This last study, was done in cooperation with Nomos Management AB, who provided their usability laboratory, via CID (Centre for Information technology Design).

Prototype implementation

Obvious from these two studies is that the PUSH project has succeeded with the other important project goal, namely to develop a prototype with useful adaptive features. During the last year we spent much time on developing an interface to the system using Java and html (Espinoza and Höök, 1996a, Espinoza and Höök, 1996b). We also spent time on making the adaptive mechanisms work as intended (Waern, 1996). The adaptive features were implemented to meet how users actually do behave when they perform tasks together with a computer system. For example, we know that users will not set up a particular goal and then stick to it for an entire session. Instead, they will continously change their goals, their plans and intentions. This reactive behaviour in the user is known within the field of cognitive science as situated cognition (Suchman, 1987; Brown, 1989; Pea, 1993), i.e. people are always acting on the situation and they will alter any goals and plans they might have as the situation changes. This is why Annika Waern implemented a forgetful, fading version of the plan recognition parts of POP - for more information on this, turn to her thesis (Waern, 1996).

In our implementation work, we also further refined the ideas for how to make the system transparent in order for the user to understand how the adaptivity should work. We use the metaphor "a black box in a glass box" to describe what we mean by this (Höök et al., 1995, Höök et al. 1996a, in press). The idea is that by making the system transparent, and to some extent predictable in how it adapts, the user will be able to control the adaptive behaviour.

During this last year of PUSH we also investigated two techniques which were not completely implemented and integrated into our prototype (yet):

Relation between cognitive ability and navigation in hypermedia

During the second year of PUSH we started a study on the correlation between individual users cognitive abilities and their ability to navigate in hypermedia. During the last year, we have finalised our analysis of the data from this study, and we have found a strong correlation between spatial ability and ability to navigate in hypermedia. Furthermore, we found that we could make distinction between spatial ability as related to spatial tasks done entirely mentally, and spatial tasks done by manipulating visual objects in the world. Only the former was related to the ability to navigate in hypermedia.

This study has been reported in (Höök et al. 1996b, Höök et al., 1996c, Dahlbäck et al. 1996). The implications from this study for design of user interfaces to hypermedia systems is discussed in these reports, but further research is needed in how to cater for the individual differences found.

Publications

The PUSH project has been very successful in getting articles published during this last year - below we provide a list of those publications. Even more important is that PUSH has generated input to two Ph.D. thesis: Annika Waern's thesis to be defended on the 6th of May 1996, and Kristina Höök's thesis to be defended on the 7th of October 1996. We also hope to see Klas Karlgrens licentiate thesis on the user conceptual understanding of the explanations provided as part of the system during the autumn of 1996. Klas has during this last year of PUSH finished his study on users understanding of SDP (the target domain for PUSH), as reported in Karlgren, (1995).

Publication list for autumn 1995/ spring 1996 (refereed articles):

Summary

The PUSH project has during its last year been able to complete the goals of the original application to NUTEK written in 1993: to develop techniques for adaptivity and to empicially establish their usefulness. We have managed to do this in a both scientifically interesting way as well as actually producing a usable prototype system.

More information on the entire PUSH project can be found in: http://www.sics.se/uacm/push.html


References

Brown, John Seely (1989).
Toward a New Espistomology for Learning. Chapter 13.
Nils Dahlbäck, Kristina Höök, and Marie Sjölinder, (1996),
Spatial Cognition in the Mind and in the World - the case of hypermedia navigation, accepted at The Eighteenth Annual Meeting of the Cognitive Science Society, University of California, San Diego, July, 1996.
Fredrik Espinoza and Kristina Höök (1996a)
An Interactive www Interface to an Adaptive Information System, , accepted to a workshop on "User Modelling for Information Filtering on the WWW" held during UM'96 at Hawaii.
Fredrik Espinoza and Kristina Höök, (1996b)
A WWW Interface to an Adaptive Hypermedia System, accepted to PAAM'96, London, April 1996.
Kristina Höök, Jussi Karlgren and Annika Waern, (1995)
A Glass Box Intelligent Help Interface, IMMI-1, Intelligent Multi-Modal Interaction-1, Edinburgh.
Kristina Höök, Jussi Karlgren, Annika Waern, Nils Dahlbäck, Carl-Gustaf Jansson, Klas Karlgren, Benoit Lemaire (1996a, in press)
A Glass Box Approach to Adaptive Hypermedia, accepted to journal of User Modelling and User-Adaptive Interaction, special issue on adaptive hypermedia, vol. 6, in press.
Kristina Höök, Marie Sjölinder and Nils Dahlbäck,
Individual Differences and Navigation in Hypermedia, (1996b), accepted to ECCE-8, Granada, Spain.
Kristina Höök, Marie Sjölinder and Nils Dahlbäck, (1996c)
Individual Differences and Navigation in Hypermedia, SICS Research Report, R9601, SICS, 1996.
Karlgren, Klas (1995)
A Study of the Understanding of Key SDP Concepts and Principles, PUSH Internal Report, available from Department of Computer and Systems Sciences, Stockholm University/KTH.
Pea, R.D. (1993)
Practices of distributed intelligence and designs for education. In G. Salomon (Ed) Distributed Cognition. Cambridge University Press.
Rudström, Åsa (1996)
On the Applicability of Machine Learning in PUSH, DSV internal report, Department of Computer and Systems Sciences, Stockholm University/KTH.
Suchman, Lucy A. (1987).
Plans and Situated Actions. The problem of human machine communication. Cambridge: Cambridge University Press.
Waern, Annika (1996)
Recognising Human Plans: Issues for Plan Recognition in Human-Computer Interaction, Ph.D. Thesis, DSV 96-004, ISSN 1101-8526, SICS Dissertation series 20.