Search All Issues, Conference Reports and Tutorials

November 1 -2, 2004
Arlington, Virginia

This conference was held back-to-back with the International Symposium on Mixed and Augmented Reality - ISMAR 04. This has long been at the leading edge of where many feel computing will go - creating new environments that one can wear, see and otherwise experience. Much of this dates back to the early efforts in VR in the late 1980’s and early 1990’s. Those days have passed but the lure and potential of the technology has not. This conference, in spite of being small, about 200, has an international scope. It is equally divided between North American, Asia and Europe.

Certainly the notion of wearable computers is not new but, having watched this technology evolve over the last 15 years, there is a sense of gradual maturity. The displays are smaller, computers are being shown that are the size of a belt buckle and many of the attendees wear a HMD throughout the conference. Certainly the interest of the military in wearable computers is a powerful forcing function, at least in the US. But what strikes us is the depth of what is considered a wearable computer and how these are being applied. We are at the beginning of a shift from computing as we know it today and wearable computers is just one component of that shift.

Bill Buxton – Time to Rethink the User Interface

Bill Baxton, well known industrial designer, yanked the audience around mentally with his notions of the user interface in the era of ubiquitous computing. His keynote address was thought provoking. It reinforced a theme heard many times before – the epicenter of personal computing is shifting from the PC.

Bill was at Xerox PARC and other research organizations. He is now on his own at Buxton Design. His keynote talk was: Whereable Computing. He made the following key points:

We are about to change the role of IO, in the human context, with computing. In the past the output, including the GUI, dominated while future user inputs will actually exceed the output from the computer.

A key to this shift is Proximal Sensing. Two examples were given to illustrate the point: automatic door opening at a hotel and auto flush urinals. The value to individuals comes when the device is able to detect proximity.

Steelcase, one of the largest manufacturers of office furniture, estimates that in 5 to 6 years it will be less expensive, on a per sq. ft., basis, to put up displays in offices than white boards. (Interesting. It would be useful to know more about how this assessment was reached given the trends that we have seen in displays and long range display costs.)

The notion of wearable technology goes well beyond wearing computers. An example was given of the Sony wrist device seen at multiple trade shows. This was cited as a device that would allow one to be recognized and secure.

The GUI is not where the growth is. It is about the relationship between devices and humans. This is where proximity is important. The social context and relationships are very important.

An interesting chart was used to show the investment in display technology. It was claimed that most of the investment in display technology is in the range of 10” to 20” but the future growth will be in both smaller and larger displays.

The trend to larger public displays was shown as an illustration of increasing presence of displays. In the context of “the society of appliances,” the social context of these displays is much different if they are seen in a “solo” context – that is, in isolation, or in concert. Specifically, if one can use the Sony wrist device to allow one to interact with the fixed display, this changes the relationship between the two devices and the wearer of the device. In this context, this is a social relationship.

One of the advantages of carrying a personal device is that it can protect one’s privacy. Again, knowing the social context and what an individual will permit protects privacy.

The notion of the social context was taken even farther. Bill cited a statement made by a famous architect – Louis I. Kahn. “Thoughts exchanged by one and another are not the same in one room as another.” The point being that social relationships are based on location. Bill used examples with the audience to show how this can happen.

Bills’ next level of discrimination was the movement of a UI from foreground to background. Examples were cited of the foreground nature of the HH interface with video conferencing and the same in the HC context with the GUI. Bill’s point in this is that we need to pay more attention to the transition between foreground and background interactions. He illustrated this with an example of Postcards – work done at Xerox PARC.

Postcards presented on one’s monitor short snap shots of individuals in their offices. This happened every 1 to 5 minutes and provided background awareness–like presence. What was found is that by using postcards, spontaneous meetings occurred between 2 or more persons. These electronic meetings typically lasted only a short time – 3 to 5 minutes. Such meetings were claimed to avoid longer scheduled meetings. Thus, the ability for an individual to go between background to foreground changed the timing and value of the social interaction.

As another example, one’s interaction in a business setting was shown with Hydra – also a Xerox PARC project. This provided 3 portals on a desk, each one being a video conference component. One could conduct a private meeting simply by approaching a Hydra portal. He was again using this to show how the social context with the HC interaction enabled new levels of interaction.

Bill used the automobile to drive the point of the car as a metaphor for the relationship of technology to individuals. He asked the question – who is more important in the placement of technology which impacts individuals in the space they create: Chris Bangle, Head of Design at BMW or Frank Gehry, the famous architect? In response, he stated Chris Bangle. Frank Gehry has no impact on what computing devices are put into the buildings he creates but Chris Bangle very much shapes your computing experience in the BMW. He deals with many companies seeking to put devices in the BMW. What happens is what he either dictates or permits.

In the car, the software which is present is quite different than what is typically taught in schools. This software is both distributed and real time. Bill also stated that electronics represents 30% of the cost of cars. Today, the second largest cost component in the BOM of a car is the wiring harness, behind the engine in cost.

Lastly Bill addressed the issue of complexity. His point is that when we take highly specialized devices which are strong in solving a problem, a few can master these devices fully to realize the strength they provide. This includes PC or even remote controls for CE. When we try aggregating multiple such devices into one large powerful solution, no one can master them. One needs to look at how a “net benefit” can be implemented to have technology master the complexity. The car example was again used where the car turns down the volume on the audio system when the phone rings. This is a simple example, but one use to illustrate how technology can be made to lessen the burdens it creates.

To show how technology can be applied to a common problem, he showed an LED projector by Symbol Technologies. This was an illustration of how a display could be made which has multiple applications where the display itself is too small to see. The example given on the podium was interesting.

Bill pushed the envelope of thinking to show how we should rethink complexity, UI and the future of computers by increasing thought about WHERE and not what.

http://www.billbuxton.com/

IBM Personal Hub – A Product that Was Passed by Technology

In 2002, IBM undertook to make a personal hub that an individual could carry with them. This had Bluetooth technology, a Linux OS and sophisticated software to make it a networking hub at which any technologist could marvel. Only one slight problem – all the functionality is being overtaken by the cell phone. What survived from the effort were the applications, especially those in health and wellness.

In 2001 the idea was compelling: a device that could interface between user appliances and a 2G/3G network. Using this network as transport, it is possible to reach rich services and content.

The hub could:

Do data logging, aggregation, filtering and pattern analysis;

The hub could route data from one body PAN device to another;

The hub could serve as a proxy for formats and protocols; and

The hub could be a personal interaction device for PAN devices and not require a display.

Many usage scenarios were shown. These included: entertainment, healthcare, commerce and travel. From this, hub design requirements were developed that included:

XML support;

Rule and event engine;

Field-upgradable software;

Display that shows alerts;

Linux OS;

Bluetooth; and

Fits in the palm of the hand.

In the end there was no such need for a custom device. The project moved to the Ericsson P800 and P900 phones. The value came from the devices created to interface to the hub – such as a blood pressure monitor. These had to add Bluetooth support; and they could then use the 3G infrastructure to reach the applications.

Intelligent Clothing

It is not surprising at a wearable computing conference one would see intelligent clothing. But what strikes us is that the dynamics of free markets is making computing-like clothing. There is low cost clothing that can be just as useful, in functionality, as expensive clothing. Elements of style and taste are more important than even function. When it is worn, it is not the same as a PC.

AlphaGrip – A 3D Keyboard and Mouse for High Speed Computing

AlphaGrip was showing a handheld device which looks much like an oversized game pad. But, it has a complete keyboard and mouse. It is claimed to have an input speed of 50 wpm. Only a wired version was shown. The device goes on sale early 2005 and will cost $95.

http://www.alphagrip.com

QBIC – A Wearable Computer Integrated in a Belt

This was a leather belt whose buckle is a computer. Sporting an Intel xScale processor and 256MB of memory, this is a complete computer that can feed a VGA display. With a Linux OS, the design was compact and impressive. Price is high at 3,250 Euro.

http://www.wearable.ethz.ch

Lumus Vision

This was an unusual HMD which looks to be no more that a transparent piece of glass in front of the eye. We saw this and were impressed. The technology to convert the display into the image on the glass is called Light-Guide Optical Element (LOE). The current display is QVGA and a VGA version is planned for 2005. Upon closer examination, it appears that the optics are segmented into what appear to be prisms. These prisms, or virtual prisms, allow the image to propagate down the glass. When looking at the display, the prism effect can hardly be seen. This is one of the least invasive HMDs we have seen.

http://www.lumusvision.com

IBM Shows SoulPad – A Licensing Challenge

It was only a matter of time. The TJ Watson Research Center at IBM was showing a device that allows for one’s machine state to be carried with them. Contained in a box about 4” X 3” X 1” this allows for the complete PC state to be moved to another computer while on the road. It uses VM technology to implement the virtual machine. The claimed value includes a simple usage model and only 2m to resume and 30s to suspend. What was shown was an early prototype. IBM did state there are a number of software licensing issues. This is an excellent example of carry vs. find.

Key Entry Product Concept Leaps Out

Ken Lyons, Georgia Institute of Technology, has developed a one-handed keyboard called the Twiddler. Here at Wearable Computers, he described training and testing to improve the key entry rate. Why would one care about a one handed key entry device – wireless messaging? Instant text messaging on mobile phones is approaching 1T per year. With input rates at only 8 w/m, there is room for significant improvement. Past efforts with the Twiddler have shown it is capable of 26 w/m and now they reported rates to 40 to 60 w/m in 80 training sessions. Such a device could represent a major improvement instant message input.

The wireless messaging market in 2002 was $12B. There are now 1.3B mobile phone users. Yet, there have been very few products that support effective text input for wireless messaging outside of the phone itself. The novice Multi-tap input on a Nokia 3210 is 8 w/m while the Motorola i85ci moves along at 11 w/m. Yet, typing on a standard keyboard can easily run at 60 w/m with burst rates to 120 w/m. If wireless messaging input is to improve, something more than a standard phone key pad is needed. At the same time, typical form factor keyboards are too large. The Twiddler is a 3 X 4 grid of buttons which is operated much like a guitar with one hand. There are one button characters and two button characters. Input is called cording.

Georgia Institute of Technology sought to improve the input rate by training users and examining how the Twiddler is used. Participants were selected that had typing rates of 54 to 114 w/m. They each spent 20 minutes per session and were paid based on speed and accuracy. The results were quite striking, with the upper rate on the Twoddler being 67 w/m and the lower rate 40 w/m. The study looked at the patterns of key usage including hold and release intervals. Also investigated was the use of multiple character strings. Another study examined blind typing – that is the use of the Twiddler without seeing what is being typed. This could even be done when the Twiddler is in one’s pocket. The blind rates varied from 67.7 to 36.6 w/m.

To be looked at the future are other key maps and means to enhance novice user performance.

I asked Ken Lyons after the talk – have you explored the use of Bluetooth on the Twiddler as a means of wireless input to the cell phone. He responded that it was a logical step.

Wrist Worn UI

Columbia University presented intriguing work on how to design an effective UI within the constraints of the very limited surface area of a wrist device. What was chosen was something which has the surface area like a watch. One of the driving factors with small area displays is that the interaction and visual attention times are short. A tablet PC has times of 1m and greater while a watch has gets only glances of 1 second. This gives rise to “Glance-able UIs.” The criteria for such UIs include:

Cursorless interface;

Consistent look and feel; and

Eyes-free for experts.

This effort began by building on the tactile watches which are on the market today, of which there is the Tissot Silent. The watch chosen for this study was the IBM Linux watch. Prototyping was done with a touch pad. The approach taken in the UI used motions of the finger on the surface of the display. This allowed the user to move through multiple levels of a menu. An advanced technique used concatenated strokes where the length of the stroke mattered.

The next step in the research is to port the UI to a Watch Pad for testing.

Public Displays that are Private

Marc Eaddy of Columbia University reported on work done with Rochester Institute of Technology that allowed one to interact with public displays and keep that interaction private. A taxonomy was presented of the input and output of public and private displays that showed the modalities used. This led to the motivation for EyeGuide. When seen on the person, the EyeGuide appears complex and burdensome. What is more important is the concept. This “HMD” like device does the following: tracks the eye of the individual watching the display. There is a camera that watches the scene and a head tracker. The device allows one to look at a public display such as a simple subway map, and the movement of the eye provides clues on the issues the observer is facing. For example, the individual may be lost and, by knowing the private use of the display map, instructions can be given to the individual. This simple example masks the level of support that such a device could provide – even in person-to-person communications with a public display.

Interruptability – It’s Relevant in Ubiquitous Environments

ETH Zurich, Switzerland, presented results on the model for studying Interruptability. This was applied in the case of wearable computers, but the concept of gaining the attention of an unfocused person to the cause of the interruption is interesting. The paper highlighted two dimensions to the problem: personal interrruptability and social interruptability. What is important here is that one does not think about the many issues that ubiquitous computing poses and, as a result, the insights were quite fresh.

WAVE Comments

What is significant here is that the envelope is being pushed on how individuals relate to computers in their daily lives. Yes, some seem silly but many others are becoming practical. Wearable Computers is an event which paints one picture of the future of computing.

One of the most important parts of Wearable Computers is the way it forces us to think about the future of computing. Yes, much of this was made evident at CHI in Vienna and again at Mobiquitious in Boston. We are seeing many more data points that begin to define the future of how individuals will relate to computers or, better yet, forget how they relate to computers. Here at Wearable Computers the automobile came up in Bill Buxton’s talk as today’s ubiquitous computing environment. This is not an open platform and, as he provided in an example, BMW defines the environment that the driver and passenger experiences. One should not forget that the most expensive component of the car, after the engine, is the wiring harness. If the car is one form of ubiquitous computing, the cell phone is the other. IBM had grand plans for a Personal Hub but the cell phone made this obsolete before the concept could even be completed.

All of this revised thinking is cause for us to face the reality that we are midst of a changing environment of how individuals use and relate to computing devices. When the PC market is flat, there is still the potential for significant growth markets. But as we saw here, and at other events, the future is not where we have been in the past. Be it a personal display no larger than stick of gum in front of the eye, a computer in the shape of a stick of gum or vehicle with more computers than we can count, the opportunity space is very different and diffuse.