Related to the project is also a study done by Preben Hansen. He is evaluating the interface to a report database named Dienst through a WWW-based empirical study. His experiences are fed into the WWW-evaluation project.
We see that information retrieval techniques and tools are increasingly getting more and more widespread and used as they are set up and used on the Internet. As we design information retrieval tools that we make avilable on the net, we are provided with a unique opportunity to get immediate feedback on the design and functionality of these tools. Instead of first designing and implementing a tool and then getting some feedback on the system after it has been implemented, we can put it out on the net and get feedback on the system from both expert designers and real users before the system is finished. In fact, we can get much more user feedback during several stages during the development cycle, starting from mock-up interface design during the early design phases, going all the way to the final evaluation and usability testing once the system is fully implemented.
This kind of scenario actually happened for an ontology tool developed by KSL laboratory at Stanford University, (Rice et al. 1995). The re-implemented the interface to their onotology tool using Netscape and html, and put it out onto the World Wide Web (WWW). The usage of their tool grew from around 100 users to about 1000 and they got rid of a lot of problems to do with making their prototype available on several platforms. What is more interesting from our perspective is that they could design and redesign the interface in repsonse to the users reactions to it while it was in use.
Continous evaluation as proposed here also consititues a challenge to existing research on methods for interface evaluation. Potentially some of the existing methods can be used in this situation as well. For example, heuristic evaluation and cognitive walkthroughs can be done equally well, and practically much more easily, if it is done over the Internet. The designer can set up a mock-up interface design slide show which expert designers are invited to look at and comment interactively over the Internet. Then these expert designers can sit anywhere in the world, and perform some of the evaluation at any point in time. Still, these methods have to be modified to cope with this particular situation.
Also the actual evaluation with real users will have to be modified to fit with a situation where we can, potentially, have access to users at any point in time.
On the practical side, we shall have to build tools that helps us in gathering and analysing the data we are provided with, both from expert designers and from users.
More information about the ideas can be found here.
Address SICS, Box 1263, S-164 28 Kista, Sweden
Tel +46 8 752 1517, Mob +46 70 56 806 35, Fax +46 8 751 7230
E-mail kia@sics.se, URL http://www.sics.se/~kia