Information Services Based on User Profile Communication

Annika Waern, Charlotte Averman, Mark Tierney and Åsa Rudström

Swedish Institute of Computer Science
Human Computer Interaction and Language Engineering Laboratory

Introduction

Web-based, user-adaptive information services impose a difficult requirement on user modeling: the inference schema for user modeling and user-adapted information retrieval must be maintained over time. There are two reasons for this: 1) the information itself may change; and 2) the user group is largely unknown from the start, and may change during the usage of the system. To address these problems, we view profile structures and information classifications as means of communication as well as means for automatic information retrieval. Users communicate their information needs, and editors communicate the structure and content of available information. Our focus is to provide tools in which such communication can be efficiently accomplished. These tools are then used to enhance the day-to-day information retrieval in terms of information filtering and search.

In this paper, we propose a design for web information services based on user profile communication. We illustrate the approach by an example application, in which conference and workshop calls are collected and distributed to interested users. We present the results of an initial study of this system, which points towards the most important characteristics of this collaboration. Finally, we discuss future extensions for the system.

ConCall: Conference and Workshop Call Reminder Service

ConCall (Waern et al, 1998) is an agent-based system that implements the EdInfo ideas1 (Höök, Rudström and Waern, 1997). The ConCall service is accessed over the web and supports the collection, filtering and browsing of conference and workshop calls. Using ConCall, the user can review calls and set up reminders for deadlines. To avoid uninteresting calls, the user sets up a user profile (essentially a filter) to retrieve a personal selection of calls and organize them in a personal manner. A user is provided with several ways of maintaining the user profile including a system-inferred candidate profile based on user interaction with the calls.

An immediate concern for the design of the ConCall user profiling was that it should be open-ended and adaptive, but yet be as simple as possible to maintain. To achieve an open-ended design, we required that neither users nor editors should be bound to any predefined ontology in the formulation of filters and the structuring of calls. Users and editors are essentially free to define their own classification schemas, although some synchronization is needed.

The filtering mechanism for the first version of ConCall relies on a particular type of meta-data annotation that we have chosen to name buzzwords. The buzzwords for a conference call are simply a set of terms that have been chosen by the editor as a useful characterization of the call. Similarly, user profiles consist of buzzwords selected from the set of editor buzzwords or invented by the users. Technically, buzzwords are treated as keywords; they form an additional parameter of the representation of calls.

The long-term aim of the ConCall system, as a demonstrator for EdInfo, is to use it as a platform in which tools for buzzword handling can be evaluated. These tools should enable users and editors to utilize buzzwords not only for information filtering and retrieval, but also as an open-ended communication channel.

User Study

The initial user studies of ConCall focussed on creating suggestions for how the filter component could be enhanced to increase communication between users and editors. The first study covered two aspects: 1) the actual set of tools that would be useful for users, and 2) the possible usage of ConCall as a peer-to-peer recommendation system rather than as a service provided by an editor.

Essentially, we wanted to know what user tools were needed for structuring calls, if users wanted to act as editors (sending recommendations) and if users wanted other users to act as editors (receiving recommendations from peers). We also sought some information on the actual quality of the service, i.e. if users were able to set up their filters so that interesting calls were retrieved. The adaptive functionality behind the candidate profile was not subject of evaluation in this study, since its mechanisms must be tuned to actual user behavior to provide useful adaptations. A secondary purpose of the study was to gather log information from usage, which will be used for this tuning.

The study was conducted through questionnaires, interview questions, and logging of user behavior during test runs with the system. The test group consisted of 11 participants, all familiar with calls for paper/participation. The buzzwords used were provided by editors only, and not tuned to the users’ needs since this was the first test performed.

A number of the functionalities examined in the study were shown to be of high interest to the participants. In particular, participants liked the reminder service, and would like to see the possibility to get recommendations, in particular from friends and colleagues.

ConCall Extensions Based on Results of Study

Since conference calls usually are distributed on a peer-to-peer basis, it seems obvious that ConCall would benefit from a scenario where anyone can add conference calls, as well as read them. It is technically very simple to do this extension. In practice, however, the success of this approach requires that some people actually will provide the calls in the appropriate structure, and seek appropriate buzzword annotations for calls. The study showed that there were individuals that are willing to take on this task. However, a critical success factor is that good editor tools are available. One way we aim to address this issue is to include tools for information extraction of call properties as well as potential buzzwords. Editors would also benefit from buzzword management tools, and tools for identifying and maintaining user groups.

A more elaborate extension is to increase the support for collaborative filtering [CACM 1997]. Users may benefit from being able to review each other’s profiles, or being presented with a typical profile for a group of users. Our study showed that an important factor here is whom you want to send the information to or receive information from.

The possibility to set up different filters for different types of information is most likely a necessary extension, as the same buzzword may mean radically different things in different contexts. Categorization also allows for better feedback to editors concerning possible annotations for calls. These categories will have a "buzz" flavor, and change over time, in contrast with most existing information services.

There were two additional wishes from users that we see as less useful additions. Users expressed a wish to increase the expressive power of profiles by adding types to buzzwords, so that the search for the buzzword is limited to a certain type of information ("Hawaii" is a location buzzword, "Johnson" is a program committee buzzword). However, it is unclear if this will have any large effects in practice: very few buzzwords are likely to be ambiguous in the sense that they mean widely different things if occurring in different context in the same call.

Finally, users expressed a wish to be able to grade the importance of buzzwords. This is a natural wish, but it is very unclear how such gradings should affect the behavior of the system. Should there be a threshold value for the overall importance of a call that determined if it would be sent to a particular user, or should the information be used locally to sort calls in order of importance? Here, we believe that further studies are needed to determine how users envision the optimal behavior of a system.

Final remarks

The ConCall service shows that edited adaptive information services indeed are possible without imposing large and complicated novel tasks on editors. Edited adaptation provides added value for the individual user as well as for the editor, who obtains better means to organize information to suit the needs of the users. The initial study of ConCall shows that there is a clear need for tools for the handling of filters. In particular, the study shows that such tools should enable recommendations, so that the individual, be it a user or an editor, can receive recommendations from peer users and editors. Furthermore, the study shows that recommendations from friends and colleagues are valued higher than any other types of recommendation.

References

Höök, K., Rudström, Å., and Waern, A. (1997) Edited Adaptive Hypermedia: Combining Human and Machine Intelligence to Achieve Filtered Information. In Milosavljevic, Brusilovsky, Moore, Oberlander and Stock (Eds.), proceedings of the Flexible Hypertext Workshop. Macquarie Computing Report No. C/TR97-06, Macquarie University, Australia. Also available at http://www.sics.se/~kia/papers/edinfo.html

Waern, A., Tierney, M., Rudström, Å., and Laaksolahti, J. (1998) ConCall: An Information Service for Researchers Based on EdInfo. SICS Technical Report, T98:04, ISSN 1100-3154, October 1998.

Communications of the ACM. (1997) Special issue on recommender systems. Vol. 40, No. 3. March 1997.