Plan- and User Sensitive Help (PUSH)

All trademarks are held by their respective owners.

PUSH - Plan- and User Sensitive Help - is a joint project with Ellemtel, SICS, NUTEK, Stockholm University, and Linköping University. PUSH is a prototype for an adaptive help assistant for a specific domain and application. We apply several methods for user- and context adaptation, and utilise empirical methods for evaluating and comparing different solutions. The application studied is a development method for telecommunication applications at Ellemtel.

The particular area of adaptivity that we adhere to can be referred to as adaptive hypermedia, a fairly newly emerging field. For an introduction to adaptive hypermedia, go here.

• A brief presentation of PUSH
• Project history
• Project information and funding
• Publications
• Project members

A brief presentation of the PUSH project

The PUSH project is devoted to finding an empirical basis for how we can integrate adaptive techniques in an information seeking system for one particular domain (the documentation of a software development method named SDP). The integration should be such that the adaptivity is acceptible to users, they must feel in control of the adaptivity at some level. Several different characteristics of the user group has been identified, and our goal is to meet the users needs through combining a multimodal interface with a partly adaptive solution.

In the project we have developed a prototype for an adaptive help assistant for users of SDP. SDP users seek information to learn SDP or to solve specific tasks in a huge online hypertext manual; they end up with massive amounts of information organized in a confusing information space. PUSH is an adaptive interface to the information in the manual. PUSH addresses the problems of information overflow with 1) a heavily domain based design of the information structure, with dynamically created follow-up questions and rhetorically typed links between explanation units; and 2) adaptive information presentation with both both plan inference and stereotypical user modeling to determine what information is presented and when.

Users pose queries by selection from situation- and context-dependent pop-up menus or the dialogue history, or by typing text input. The set of queries PUSH can handle have been determined through empirical studies of SDP users and their information needs - a corpus of user queries to SDP experts has been collected - but since help request almost by definition cannot be anticipated, the system must be able to deal with unexpected user queries and needs. The text processor accepts any input, and maps them on to the set of internal queries; if no exact match is found, the least general query that is consistent with the input analysis will be chosen. This means that users may well be given answers inconsistent with the gist of their queries; this will still give the user an idea of what PUSH handles, better than error messages would.

A prototype system has been developed. It utilises www and Java as its interface. The user's actions are monitored by the system and the presentation of information is affected by the inferred task of the user. The adaptive system has recently been evaluated in a comparative study (see also "Evaluating adaptive system" for a summary on the difficulties in evaluating adaptive systems. The results are not yet described, but the subjects evaluation clearly indicated that the adaptive system was preferred over the non-adaptive system.

Unfortunately, the prototype cannot be made accessible for public use since the method described in the database, SDP, is proprierty to Ellemtel.

Project History

PUSH first year (Sept.1993 - Sept. 1994)

PUSH second year (Sept. 1994 - Aug. 1995)

• one was a task analysis based on Cognitive Task Analysis by Roth and Woods. It started from the needs fulfilled by the current on-line manual and attempted to find the information seeking tasks of the users. The work has been reported in Malin Bladhs MSc thesis and a paper together with Kristina Höök was accepted at the Interact conference in Lillehammer 1995.
• another study focused on the cognitive differences between users and their understanding of the graphical parts of the current on-line manual. The hypothesis was that there is a relationship between spatial ability and the ability to navigate through graphs. The study is not yet completed, but some interesting results can already be noted. For example, the users were unexpectedly unsure of whether the information they had found in a graph actually was the correct answer to a question posed by the experiemental leaders.
• following the ideas presented by Micki Chi, we tried to identify the relevant categories that would describe various concepts in SDP, and furthermore, which of those concepts that would cause problems to the SDP learners. The hypothesis is that concepts that are initially placed in the wrong category by the learner, will be harder to learn the proper understanding of - a so-called category shift must occur -, than concepts which are placed in the right category but perhaps with a slight misunderstanding of the exact definition of the concept. This study is not yet finished, and results so far indicate that this domain has properties which makes it very hard to analyse the learners understandings and misunderstandings of concepts.
• finally, we performed a small evaluation of the first prototype produced during the first year. The users were positive to the prototype and some interesting viewpoints came up during study. For example, users indicated that they would like to know what other users had looked at and found useful.

During the second year we continued the work on designing and implementing our ideas. A second prototype was developed. It includes the following interesting characteristics:

• We started to develop a weak natural language input component, exploring the ideas developed by Jussi Karlgren.
• A plan inference component, which infers the information seeking task of the user (as found in our studies) from the dialogue with the system. The ideas have been developed by Annika Waern.
• A user controlled alternative to the more adaptive plan inference part has been built by Kristina Höök in cooperation with Malin Bladh.
• We started to develop an interactive html-solution that builds html-pages on the fly as a result of the interaction with the user. Fredrik Espinoza studied the problem of how to build interactive html scripts.

PUSH third year (Sept. 1995 - July 1996)

During the last year of PUSH we have finalised the implementation of the prototype and performed two studies of the system. The first study aimed to bootstrap the adaptive behaviour of the system and give an initial evaluation of the interface (and is described in a recently accepted contribution to PAAM'96: "A WWW Interface to an adaptive Hypermedia System".

We have recently performed the comparative study of the acceptance of the adaptive behaviour of the system. Our aim is to look at whether users feel in control of the system, if they can understand the inner workings of it at some level (transparency) and whether they can predict its behaviour. The results from the study has not been analysed yet, but the subjects own evaluation of whether they preferred the adaptive or non-adaptive system shows that they clearly preferred the adaptive system.

The annual report to NUTEK has just been completed - you can find it here.

Project information and funding

During the last year DSV, IDA and SICS are all funded by NUTEK on two different grants.

The first grant (350.000:-/year) which finanses DSV and IDA is funding the following activities:

• the study on the correlation between spatial ability and navigation in hypermedia started during the second year. A SICS research report has been written (see below) and scientific papers have been submitted to different conferences. Since the results were so intriguing, we are planning to follow-up on the results and might do an application for making another study to see how we can improve navigation to better fit all users, both with low and high spatial ability. Involved researchers have been Nils Dahlbäck, IDA, Kristina Höök, SICS, and Marie Sjölinder, DSV.
• a second study on how the basic concepts in SDP are learnt by novices and experts. The study is based in theories on learning and aims at finding a good basis for how to structure the information in the database of the PUSH system. The study has been performed by Klas Karlgren, DSV, who aims to complete his Ph. Lic. on this and related issues.
• an implementation of a free form query interface to the PUSH system. The implementation was started during the second year of PUSH and will be finished during the autumn and early spring. The implementation work is done by BSc Stefan Zemke, DSV, with supervision of Jussi Karlgren, SICS, and Kristina Höök, SICS.
• an evaluation of the potential use of machine learning in the PUSH system. Machine learning would potentially make the PUSH system more flexible in its adaptation to users. The ideas will be studied foremost by Åsa Rudström, DSV, with some assistance of Annika Waern, SICS/DSV (both are funded on other resources within DSV but they are part of the project).

• an implementation of the PUSH interface in World Wide Web (www). PUSH requires an interactive interface, and www has up til now only offered a rigid interaction style. A challenge has therefore been to extend the possibilities offered by www. The implementation work is done by Fredrik Espinoza who is completing his Masters degree on his work. The work has been supervised by Kristina Höök, SICS.
• a study of the non-adaptive parts of the PUSH system aimed at evaluting the interface of the system. The study is performed by Kristina Höök, Fredrik Espinoza, SICS, with some assistance of Klas Karlgren, DSV.
• a study of the acceptance of the adaptive behaviour of the PUSH system (see above).
• a Ph.D. thesis by Kristina Höök, SICS, on the design of the adaptive hypermedia system in PUSH.

Publications

• "Adaptive Help by Navigation and Explanation", Benoit Lemaire, Catriona McDermid, and Annika Waern, presented at a Swedish workshop on Explanations in Linköping, 1994 (available from SICS).
• "Satisfying User Needs Through a Combination of Interface Design Techniques", Malin Bladh and Kristina Höök, In Knut Nordby, Per Helmersen, David J. Gilmore, and Svein A. Arnesen (eds.) Human-Computer Interaction, Interact'95. Chapman and Hall, Oxford, England, 1995.
• "A Glass Box Intelligent Help Interface"(ps), Kristina Höök, Jussi Karlgren and Annika Waern, IMMI-1, Edinburgh, 1995. An edited version ready to be printed in the WWW-publications of IMMI can be found here(ps)(Z).
• "Intelligent Design of Help rather than Intelligent Help?", Kristina Höök, Malin Bladh, Jussi Karlgren, Benoit Lemaire, and Annika Waern, SIG meeting during UM conference in Boston, 1994.
• "Plan Inference for a Purpose", Annika Waern, UM'94, Boston, 1994.
• "Cooperative Enrichment and Reactive Plan Inference - applying plan inference outside natural language dialogue", Annika Waern, SIG meeting during Um'94, Boston, 1994.
• "Using Surface Syntax in Interactive Interfaces"(ps), Ivan Bretan, Niklas Frost, Jussi Karlgren, Paper presented to The 10th Nordic Conference on Computational Linguistics, Helsinki, 1995.
• "Mumbling - User-Driven Co-operative Interaction"(html), SICS Tech. Report T94:01, SICS, Sweden, 1994.
• "Inferring Complex Plans"(html), Kristina Höök, Jussi Karlgren and Annika Waern, poster presentation at 1st IWIUI, 1993.
• "The Glass Box User Model for Filtering"(ps), Jussi Karlgren, Kristina Höök, Ann Lantz, Jacob Palme and Daniel Pargman, poster presentation at UM'94, Boston, 1994. (There is also a long (html) version.)
• "An Interactive www Interface to an Adaptive Information System"(html), Fredrik Espinoza and Kristina Höök, accepted to a workshop on "User Modelling for Information Filtering on the WWW" held during UM'96 at Hawaii.
• "A Glass Box Approach to Adaptive Hypermedia"(ps), Kristina Höök, Jussi Karlgren, Annika Waern, Nils Dahlbäck, Carl-Gustaf Jansson, Klas Karlgren, Benoit Lemaire, accepted to journal of User Modelling and User-Adaptive Interaction, special issue on adaptive hypermedia, forthcoming, 1996.
• "Adapting explanations to the User's Task"(ps), Kristina Höök, SICS Research Report R95008, SICS, Sweden, 1995.
• "Individual Differences and Navigation in Hypermedia"(ps), Kristina Höök, Marie Sjölinder and Nils Dahlbäck, In Proceedings of the eigth European Conference on Cognitive Ergonimics (ECCE-8), 1996.
• "Individual Differences and Navigation in Hypermedia"(html), Kristina Höök, Marie Sjölinder and Nils Dahlbäck, SICS Research Report, R9601, SICS, 1996.
• "A WWW Interface to an Adaptive Hypermedia System"(html), Fredrik Espinoza and Kristina Höök, accepted to PAAM'96, London, April 1996.
• "Spatial Cognition in the Mind and in the World - the case of hypermedia navigation", Nils Dahlbäck, Kristina Höök, and Marie Sjölinder, accepted at The Eighteenth Annual Meeting of the Cognitive Science Society, University of California, San Diego, July, 1996. (This file is now complete - we hope!)
• "A Glass Box Approach to Adaptive Hypermedia", Kristina Höök, a Ph.D. thesis put forth at department of Computer and Systems Sciences, also in SICS Dissertation Series number 23, 1996.

Project members

The PUSH research group (currently) involves:

• Kristina Höök, (project leader) SICS
• Nils Dahlbäck, Linköping University
• Måns Engstedt, Ellemtel
• Fredrik Espinoza, SICS
• Carl Gustaf Jansson, Stockholm University
• Jussi Karlgren, SICS
• Klas Karlgren, Stockholm University
• Åsa Rudström, DSV, Stockholm University
• Annika Waern, SICS
• Stefan Zemke, DSV, Stockholm University

• Anna-Lena Ereback, SICS
• Catriona McDermid (now at Telia Research AB in Haninge, Stockholm)
• Benoit Lemaire (now at Universite Pierre Mendes, in Grenoble, France)
• Malin Bladh (now at Softlab AB, Stockholm)