Mark Tierney
E-mail: mt@broadcom.ie
Åsa Folkeson
Telia, Sweden.
By using newly emerging Intelligent Agent technology, the project seeks to provide the users with a Personal Services Assistant (PSA). A PSA is a form of mediation between the users and the services, and it operates on behalf of a user in an analogous manner to a human personal secretary. To encourage the public to make use of the kiosk-based services, it is extremely important to develop a system that is easy to understand and easy to use. All too often, information is not easy to find, not comprehensive and not in an understandable form.
Usability is now a very important issue within software development. Frequently usability problems are not identified until a system is already implemented. This results in systems being delivered that do not satisfy the users needs, or are not easy to use. Users commonly complain that they cannot navigate the system to find information or have difficulty understanding the information when it is presented to them. In the majority of cases the system does not meet the users expectations.
What this in effect means is that to design something that is usable, we must understand the context in which the system will be used. We need to know the type of users who will use the system, the nature of their work, the tasks they wish to accomplish and the physical and computing environment within which the tasks will be carried out. But, for a product to have a high degree of usability, it must also accommodate human cognitive characteristics. In other words the design must be adapted to the users ability in interacting with technology, such as memory, perception and problem solving skills. Regardless of the product type, designers have to take the whole product into consideration and ensure that the final product is adapted to these cognitive characteristics.
The understanding of how humans interact with technology is called ergonomics. This means that whether we are building a high performance jet fighter or a reference manual for a new wheelbarrow, failure to consider the human cognitive characteristics can result in an unusable product. For example, if we design an aircraft to bank and corner and it places twenty G's of force on the human occupant, the pilot passes out and the plane crashes. Similarly, print the wheelbarrow reference manual in four point print, and they cannot use it on the manufacturing floor without a magnifying glass.
To succeed in the exertion of producing and designing a user friendly system, Usability Tests and Field Trials with real users will be conducted during the KIMSAC project, where the context within which the system will be used will play a major role. As already stated, the context can affect the usability of a system significantly, and therefore the development team, together with Social Welfare Services and FÁS, focus on this by specifying different key characteristics of the context (e.g. user categories and nature of the tasks), that could be critical to a successful use of the system.
Within the KIMSAC project, the evaluation was performed by usability experts, who checked the user interface of a partly working prototype, to verify the level of compliance with acceptable ergonomic guidelines (standard Human Computer Interaction usability guidelines). The method was used to discover the usability problems in the user interface design by assessing different evaluation criteria, for example explicitness of the information displayed to the user. These criteria could then be a part of the iterative design process. As a result of the walk-through, specific comments and design recommendations to individual screens in the system were made.
This method cannot replace usability tests with real end users, as some of the problems revealed in usability tests might never turn up during heuristic evaluation. Furthermore, there may also be problems discovered in heuristic evaluation, which would have been hard to find by usability lab testing or field trials with end users. Walk-through evaluation, is a rather quick and inexpensive method, by which the most significant usability problems can be found.
Within KIMSAC, the selection of intended users (or subjects) was made by Social Welfare Services, who based the selection on a range of characteristics such as education, computer experience and length of unemployment. This group of people were asked to use the system, perform a set of tasks, and identify problems that they encountered during the test session. There was always a test conductor in the test room with the subject and trained observers observed the test session from an adjoining room behind a large one-way glass screen. Videotapes, audio recordings and other data were used in the lab environment to gain additional insight (see Figure 2).
The KIMSAC videotapes were analyzed after the test, and the problems revealed in the sessions were grouped and described in an evaluation report, where recommendations for improving the system were made. Because of the real user trial nature of the project, it is also - apart from the findings during the test sessions - important to pay regard to the users attitudes and expectations about using the prototype. Therefore, the users were asked to fill in two forms, one "Pre-Questionnaire" before the test started, and one "Post-Questionnaire" after the test had been conducted, to show if and how their attitudes changed during the session.
The Usability Laboratory test showed that, at this stage, all users experienced some troubles using the system, especially difficulties concerning navigation and help options. But despite these problems, an initial apprehension of using a new technology and the unfamiliar nature of the test environment, users generally reacted positively towards the system. In particular, they liked the idea of being able to find information for themselves!
To obtain usable and correct results from the tests, the tests should be run by Human-factors professionals3 who keep developers focused on the users and save the project from expensive trial and error. Inexperienced examiners often ask leading questions during the tests, give the user clues with body language, use the terminology of the application, and misinterpret the results.
By involving the users in an early stage in a multimedia development project, by taking the whole context into consideration and by designing the user interface iteratively, the possibility to succeed in developing a user friendly and cost effective product increases considerably.