This presentation reports work being done at SICS on novel interface techniques to large projection environments (CAVE-like environments such as the SICS Grotto)intended for multi-user applications. We have been exploring the notion of "naturalness" as it relates to interfaces. We use the term interface in a restricted way, covering the gap between the user and the multi-user virtual reality platform DIVE, i.e. instantiations of different techniques of interaction within a virtual world. The goal is to create interfaces which are compelling while at the same time, more effective and less encumbering than merely extending the standard 2D GUI tools.
Designing such interfaces presents a series of design choices, centered on granting user control, number of degrees of freedom to be presented and how large to make the human-computer 'language.' We have set out to make these interfaces as "natural" as possible. We are far from final conclusions, but are instead presenting here a number of our experiments.
The SICS Distributed Interactive Virtual Environment (DIVE) is an experimental platform for the development of virtual environments, user interfaces and applications based on shared 3D synthetic environments. Dive is especially tuned to multi-user applications, where several networked participants interact over an internet.
Navigating with the full six degrees of freedom has shown to be
difficult for people not accustomed to a 3D mouse. When using the scrolling
mechanism, navigation is restricted to a horizontal plane (often used with
gravity). The avatar, and thus the viewpoint, is moved forward or backward
when the cursor, controlled by pointing with the 3D mouse, is located above
or below the three screens. The travel velocity is related to the distance
the user points outside the screen boarders. In a similar way, the user
rotates by pointing off the screen to the side.
The ball navigation technique is implemented using a magnetic
tracker attached to one side of the ball (an ergonomic sitting ball, approximately
80 cm in diameter). A user sits on the ball and leans in the desired direction.
At the same time as the user is navigating, the user can employ the
3D mouse to select and grasp objects. This is an attempt to reduce the
workload of the hand. This navigation method seems to be intuitve as it
appeals to an everyday notion of locomation as well as being less fatiguing
than other more active mechanisms.
In order to give a greater feeling of depth, we have implemented a "flying
cursor". The flying cursor, a flat partly transparent hexagon, is placed
on the surface of an object that is being pointed at, and tilted according
to the normal of the surface. This provides extra spatial cues, allowing
the user to 'feel' the surface of an object as well as get another depth
cue through perspective.
Unencumbered Video Tracking
This is perhaps the most powerful and least encumbering of all the devices.
Having used magnetic trackers for most interaction techniques, one realizes
that it would be an improvement to unencumber the user of cabling and sensors
Toward this end we have implemented a navigation technique that uses the
Pfinder system developed at MIT Media Lab. Pfinder extracts information
from video images of a person in a scene, and gives 2D+ data about the
location of the hands, head, and center of the person's body. This information
can be used to navigate and manipulate objects in a VE. We have implemented
a way of navigating using a simple body gesture command language. Raising
your arms increases the speed, lowering one of your arms makes you turn
in that direction. This navigation is based on the metaphor of "gliding",
much the way a gull might turn and bank in the air.
The trampoline navigation interface uses an ordinary workout trampoline
with a magnetic tracker attached to the bottom. It is controlled in a similar
way to the ball and is an attempt to take some of the interaction away
from the hand. The user stands on the trampoline and leans towards the
direction he wants to move. This technique has, as well as the ball technique,
proven to be intuitive (almost like a surfboard/skateboard). However, the
current implementation has proven quite hard to characterize and can therefore
be straining for the leg muscles as it works with very small angle changes.
A disadvantage of this interface is that it is very sensitive to natural
changes in position, like changing the weight of your body to another foot.
Path Planning Map
In a virtual environment, one way to let the user gain a larger context
of the environment is by using the WIM (World In Miniature) metaphor, which
in our implementation is called a Path Planning Map (PPM). This provides
the user with a hand held miniature representation of the world, somewhat
like a map in the real world. The virtual map metaphor benefits from the
notion of everyday maps, something most people are familiar with, and thus,
a concept that can be grasped quickly. The PPM can be used for locating
and orienting the user, searching for other users, and for moving directly
to a point on the map.
One mode lets the user draw a path on the map, which will be replicated
in the virtual environment and displayed to those that choose to see it.
A user can follow the path freely or be taken on the path automatically.
A tracker can also be attached to a physical plate directly linked to the
virtual map, allowing the user to hold a real representation of the map
at the same time as controlling the virtual PPM. [This work
is done in conjuction with Emmanuel Frécon.]
When constructing this interface we looked at how a tank driver steers
a tank. The two wands control the speed of the virtual tracks. Magnetic
trackers are attached to two sticks to simulate tank navigation. Rotating
both sticks forward will give you a forward velocity. Rotating the left
stick back makes you turn left. The directional velocity is determined
by averaging the sticks rotation and the angular velocity is computed by
looking at their rotational difference. An advantage of this interface,
apart from being intuitive, is that it only uses 1 degree of freedom (pitch)
which prevents a beginner from producing unexpected movements. The fewer
degrees of freedom an interface uses, the shorter the learning period needed
since the 'vocabulary' is limited. However, a very limited 'vocabulary'
prevents use of the interface in a general sense. The 'Tank' interface
is appropriate for navigational purposes. Using the base mechanism of the
angular control sticks, the tank vehicle is extendable to other vehicles
such as a virtual fork lift. The shifting of mode would depend on the application.
To achieve more direct manipulation and more natural environments (in
the sense of manifest affordances), we are working on novel projection
displays in connection with touch screens as a method of achieving alternatives
to 'fishbowl' interfaces. The above photos illustrates one such setup where
we have a horizontal 'desktop' with a touch screen interface, and a vertical
'communication wall' that we can use for conferencing. Both surfaces provide
an interface into the virtual environment/workspace. The horizontal
projection surface also provides an elegant mechanism for local groupwork.
In the image on the right, the desktop is used in conjunction with the
multi-screen display to provide a view of the virtual desktop which is
located in the VE. Objects on the desktop can be manipulated by direct
A tracker is placed on a puppet's body and another in it's right arm. The puppet is then used as a control agent in the VE. In navigation mode, by turning the puppet, the avatar's position, and thus view, changes accordingly. The tracker in its arm is used for specifying direction, velocity or manipulating objects. A microphone, placed in the puppet's ear and used with voice recognition, and a speaker in its mouth, encourage the illusion of an embodied virtual agent. There are many extension applications that can build on the puppet-agent technology - such as a tour guide in the VE. When picking up the puppet in the CAVE-environment, it would know what world you're in (and where you are), you could then ask it to take you where you want to go. Along the way, the puppet¹s arm could be used to point at interesting objects and questions could be posed.