Search All Issues, Conference Reports and Tutorials

August 23 -25, 2004
Boston, MA

Professor Mahadev Satyanarayanan, Carnegie Mellon University, gave the keynote address and proposed a concept for ubiquitous computing called “internet suspend/resume.” He is also a director at the Intel Research Labs in Pittsburg. He asked the question - Will the PC whither in the era of ubiquitous computing? In response, he suggested that PC mobility has been achieved when one can move from location to location and log onto a different machine and each one preserves one’s personal state. Each computer in such a mobile environment is identical to the last computer visited and it has all the characteristics of “my” machine. One’s computer experience is thus cached over the Internet and the machine can be suspended and resumed on any connected computer.

Having attended the 13th IST Mobile & Wireless Communications Summit in June, the WAVE got a sample of the European view of mobility and a glimpse of its view of ubiquitous computing. It is different here at Mobiquitious. If there is an underlying thread here, it is the vision of Mark Weiser. Yet, going from vision which sees computing fitting this paradigm

“The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.”

to reality is one large leap. In fact, Professor Satyanarayanan called this a vision driven agenda. But getting from the vision to reality is the hard part. There were far fewer clouds on the horizon in Europe.

Seamless Mobility on Ubiquitous Hardware

Professor Mahadev Satyanarayanan gave his talk on the topic Seamless Mobility on Ubiquitous Hardware in both his affiliations at CMU and the Intel Research Center in Pittsburgh. There is a project on Internet Suspend/Resume (ISR) on the CMU campus. The presentation touched on many issues in Ubiquitous Computing.

In the 80’s and 90’ the concept of mobile computing was the ability to connect up virtually anywhere with a light weight portable computer. In practical terms with WiFi and lightweight notebooks this has happened. We have arrived at the Eden of Mobile Computing.

From the computing hardware perspective, this has been a technology driven agenda. The computer has gone from the luggable Compaq of 1997 to an IBM Linux Wristwatch in 2000. Communications has gone from a NCR WaveLan card in 1990 to Centrino chips in 2003. The progress in hardware has been excellent.

Yet, many challenges remain. This has been the focus of much research. These core challenges include:

Resource Poverty

The notebook will never be able to match the resource capabilities of what is present on the desktop computer and with its power sources and connectivity.

The capabilities in mobile are wimpy compared to the desktop.

Communications Uncertainty

There is a huge dynamic range of connectivity. The issues center on bandwidth, latency variation and intermittent connectivity. A significant barrier is that connectivity may cost real money.

Finite Energy Source

The limitations on battery power mean that actions may be slowed or deferred. The practical matter is that communications are one of the major consumers of energy.

Multi-Modal Interaction

Mobility also means that one’s eyes and hands are frequently occupied with other tasks. As a result, other means of interaction can be useful. This includes speech and gesture recognition. I expect that augmented reality may have a major impact.

Scarce User Attention

Because one’s focus is frequently elsewhere, there is lower human performance and higher human error rates. Limited attention span is like a dumb user.

Less Security and Robustness

There is a conflict between portability and size. Thus, as with the cell phone smaller is better. As a result, these are frequently lost and broken. At the same time the likelihood of subversion increases.

As we look to Ubiquitous Computing, a.k.a. as Pervasive Computing, there is one person responsible for this vision – Mark Weiser. His seminal 1991 paper in Scientific American defines his views and the vision of computing into the 21st century.

Compared to the hardware which enables mobile computing, Ubiquitous Computing is driven by the vision that Mark articulated. As a result the issues include:

If hardware is almost free, what is expensive?

How does one further reduce the cost and how is this new world created?

This has led to new areas of research in context-aware computing and smart spaces.

At the same time this has become a major area of investigation in HCI (see the report on CHI 2004).

In assessing the trades between being portable vs. Ubiquitous, we must examine if a resource is carried or found. Simple examples were used to illustrate the role of portable vs. ubiquitous. Air is ubiquitous and we seldom need to carry it with us. Water is almost of the same situation but we do carry more water with us, especially as a function of the geography such as foreign travel and hiking. When it comes to personal computing, portable using notebooks has become the mainstream computing platform compared to desktop only systems. There are no ubiquitous personal computers in this context – such as the thin clients. The question then is – what will the future computer look like when there is a trade between carry and find, i.e., portable vs. ubiquitous?

The bottom line is that ubiquity can be seen as a substitute for portability.

As we look to the future of personal computing in this mobile world, the role of applications and the level of personalization of our computer plays a very important role. The computer we use is “our” computer defined by how we have customized it.

To realize the importance of this, we must look back to the future to the era of time sharing in the mid-60’s to the early 80’s. This was a period of time sharing – what we would call today thin clients. These thin clients, were on dumb terminals and our state was held at a central mainframe. But it had perfect user mobility access across points of use. Our personalized computing followed us.

Enter the era of personal computing. Initially we won big on performance and usability (windowing interface) but we gave up on mobility (as in the context of the time sharing user). What has happened is that one’s PC has become a personal fortress. We live in our own computing world as defined on the PC we use. Moving to another computer, when we sell an old one, is an arduous task. This clearly discourages the use of pervasive hardware.

The questions then are:

How can we enjoy seamless mobility across pervasive hardware?

Avoid painful re-customization and the resultant loss of productivity?

Enter Internet Suspend/Resume (ISR). The inspiration is to make access to one’s personal computing state perfectly seamless. Thus, any computer becomes “your computer.” Your entire personal state is centrally managed.

The strategy is based on having a layered virtual machine on a distributed file system. This has the following features:

The virtual machine encapsulates the state cleanly;

The distributed file system transports state via caching;

State is considered precious and safe on central servers and

Local caching is permitted.

The ISR architecture has the user environment, such as MS Office and the guest OS, such as Windows XP, at the top of the environment. This is, what the user sees. On the client is a virtual machine which creates the virtualized hardware to support the guest environment. There is then a glue layer which runs on a bare OS using Linux. The file system is Coda.

This implementation is an excellent example where the usage complexity is low and there is a high level of internal complexity. By low usage complexity, it means that the user sees the same environment they are accustomed to. The difficult part is getting this to work across many target computers.

Caching plays a very important role. Examples were given of early tests on Resume Latency using both multiple means of caching and with various bandwidths. Some of the simulations showed resume times of only 12 seconds across bandwidths from .1Mb/s to 100Mb/s.

ISR is an experimental prototype. It is running on the CMU campus. The system is characterized as reasonably robust but not of production quality. It will become a small scale pilot deployment on the CMU campus.

This paper extends the concept of virtualization to mobility. Very interesting.

Is Ubiquitous Computing Here Today?

Here at MobiQuitious we are surrounded by presentations on these technologies:

Ontologies
Semantic Tuple Spaces
Ad Hoc Networks
Service Discovery
Service Networks
Semantic Networks

and more. Yet, everyone admits that the foundation for this activity is the original work of Mark Weiser. He wrote in 1988 from his position in the Computer Science Lab at Xerox PARC:

“We call our work "ubiquitous computing". This is different from PDA's, dynabooks, or information at your fingertips. It is invisible, everywhere computing that does not live on a personal device of any sort, but is in the woodwork everywhere.”

Yet, it seems that the elegant concepts of Mark Weiser have been turned into a research agenda which misses the point. Are we not seeing ubiquitous computing today and fail to recognize it?

What is Ubiquitous Computing?

Tim Finin, (UMBC) also gave the only perspective of what makes ubiquitous computing. His points included:

(1) It exists in an open, heterogeneous and dynamic environment;

(2) There is a high degree of cooperation

(3) Context sensitive functionality

(4) Personalization is driven by user models and data and

(5) It has AI like capabilities.

He then drew the conclusion that a service view of ubiquitous computing is very appropriate. It is not possible to do this on a cell phone or a wearable computer. From this he drew the conclusion that heavy use will be made of software agents and semantic web languages.

The WAVE asked the question:

Do we not already have a ubiquitous environment in today’s automobile, especially the high end models which have 100 + processors?

The environment here performs many services such as:

Dashboard control;
ABS;
Engine Control;
Lighting;
Ride and
Telematics.

The driver is not even aware of all that is going on. The processors are networked and getting more so.

Increasingly the automobile network is being tapped externally, at a minimum, to do support repair operations.

Thus, are we not imprinting our own technologies and research agenda onto ubiquitous computing and as a result missing where it is happening today?

Response: Essentially no answer.

Optimal Bluetooth Scatternet Formation

Leigh Hodge of the Center for Mobile Communications, Cardiff University, Wales, UK presented an approach to optimize the performance of Bluetooth Scatternets. A scatternet is made up of one or more piconets. Each piconet can have up to 8 devices and occupies a star configuration. Bridge devices, i.e., nodes, are in multiple piconets. There are two types of bridges: SS – slave in all piconets and MS – master in one piconet.

The Bluetooth spec permits scatternet operation but the protocol remains open. There have been many protocols proposed and an extensive survey will be published shortly in Ad-hoc Networks Journal. One of the shortfalls is that the techniques for measuring and comparing protocols are not clear.

This paper gave a preliminary view on how it will use a global awareness of the device location and to provide an absolute benchmark. A genetic algorithm is used to address this as a multiple objective problem. The algorithm selected is NSGA-II. One of the challenges is to represent the scatternet so that it can be optimized and not violate the rules of the scatternet.

The research plan is on going. Due to the high levels of computation for the optimization, and the intent to analyze 1,000 networks, the simulation time is expected to take 2 months. The results will be made available over the next 12 months.

Hand as HID (Human Interface Device)

Tobias Hollerer, UCSB, examined how hand gestures can be used to control devices and computers. He began with the premise that a mobile computer can do many things and, to illustrate, showed many devices including a cell phone, a digital camera and computer game. One of the problems is that wearable devices, a mouse included, has a small interaction footprint. However, the hand has a much larger range of movement. He called this vision-based interfaces for mobility. Required to observe the hand is a camera near the head and eyes. The advantages cited include:

Hands are free with no need for gloves or tracking devices;

Lightweight, mobile and potentially wireless;

Works well in adverse environments;

Camera use for hand detection can be used for other purposes
and

Affordable.

Three types of interaction were described:

Registered – pointing to a displayed object or space.

Pointer Based manipulation – interaction with a displayed object. (Similar to direct manipulation)

Location-independent interaction – hand gestures to determine actions (6 hand gestures are recognized)

The detection of these hand gestures is accomplished with the API called HandVu. It typically takes 5 – 150ms to interpret the hand on a 3GHz processor with a 640 X 480 sensed area. The current performance is felt to be acceptable but it does require considerable computation. The process steps go from hand detection to hand tracking to posture recognition. The detection rate is 92% and with a false positive rate of only 10**-8. One aspect responsible for the low false position rate is the use of color for hand detection.

Do We Need ContentCascade?

A product of Georgia Tech in Atlanta and the HP Labs in Palo Alto, ContentCascade is a means to use public displays, such as store signs, assumed to be digital displays, and interactive ones at that, to push content to passerby’s. If the passerby shows interest by some interaction, a dialog would ensue with the individual. As the paper said, the problem statement includes the following relative to public displays, i.e., signs:

The amount of interactive content exchange is limited due to short attention span, privacy issues and resource constrains due to what the individual is carrying. This is compounded by limited human memory and the fact that much of the viewed content will be forgotten.

Examples of what can be downloaded to the user included:

URL of a web site Thumbnail of a poser cover or page of a book or Small movie trailer.

What was missing from this presentation was:

Does the consumer care?

If such content can be pushed to a cell phone or ubicomp device then the burden lies with the consumer to set up rules on what will be permitted or not. The scenario then has the consumer “programming” the device before coming to the mall to look of offers of shirts, for example, when walking the mall. Who needs such a burden?

Augmented Reality and Ubiquitous Computing – Another Non-Starter

Eiji Tokunaga of Waseda University in Tokyo, Japan presented concepts for how augmented reality, a.k.a. mixed reality, could be made useful to individuals and consumers. Basically, this is a means to combine virtual objects on a real world visual space. The tool kits suggested for use include ARToolkit from the HIT Lab at University of Washington and Trip from the Sentient Computing Lab at the University of Cambridge.

Examples were shown of:

Using a PDA in a mobile reality set up to determine the location of a refrigerator.

Configuring a control device, such as a PDA used for remote control of a television set, based on having the device recognize the television; and

Performing gesture recognition as HID.

When asked will this work in ad hoc environments, the response was no.

Take away - high levels of complexity with little in the way of output.

Rule Processing Applied to Wearable Computing – The Computer Has NOT Disappeared

Masakazu Miyamae of Osaka University described a number of examples of wearable computing using ubiquitous computing technologies. Using a services-based approach, when a user approaches a facility, navigation is provided – using an HMD as the interface. The rule engine is based on Events, Conditions and Action. In the prior example, the user moves near a building and the display shows the name of the building.

To show how this system works, examples were shown of a video navigation system, a tour of a park, a farm work system and a motorbike race support system. In the case of the farm work system, the workers would wear gloves and on one finger was a temperature probe.

Yes, this is all research. But here is technology that is both obtrusive and of marginal value beyond what can be accomplished by other means.

Migrating Applications based on Location

Anand Raganathan of the University of Illinois, Urbana, described a simple problem:

How to migrate an application, in this case Powerpoint, for a presentation, into multiple rooms which have various degrees of support for the application?

For example, if one enters a room which has only Linux computers the application would create a pdf form to show. The concept is interesting and a variation on the theme expressed by Professor Satyanarayanan earlier. That is, does one carry or find in ubiquitous environments? In this case, the burden on adaptation lies with the user to make the task work in what has been found in the new environment.

In this example, why not carry the notebook?

Migrating User Interfaces

Donatien Grolaux of Catholic University of Louvain presented the notion of migrating portions, that is, screen portions, of a user interface from one device to another. For example, a control panel can be moved to a PDA and seen as one walks a factory floor. Another illustration, where there is a multi screen display of sensor output and one sensor to be monitored, can have its display migrated, also to a smaller screen device.

Interesting variation on the theme of mobile applications but applied to the context of the display real estate. The only problem is that the software used Mozart-Oz and this requires that all applications be rewritten using it.

WAVE Comments

Will Ubiquitous Computing Degenerate to Personal Spam on Legs?

Listening to multiple presentations, we struggled with “where is this going?” One of the presentations on ContentCascade, above, described how individuals can be informed on public displays and then interact with them. It then sank in that many of the services are about pushing information, in part context sensitive, to individuals as they passed by a source. The rules are the means to determine what content and to assess the interest level of the individual. One motivation for these complex environments is to set up rules based on individual preferences, geography, the access device and at the same time gauge potential level of interest. We were left cold. The ContentCascade paper would have us walking in a mall with a PDA or cell phone. If one showed the slightest interest, or possibly not, the consumer could be bombarded with “deals” or enticements to come to one store or many stores. It looks like spam on legs. The problem is that the burden shifts to the user to construct rules on what is acceptable and not. If ubiquitous computing reduces to this, it is not clear who would want it.

More

This conference is at the convergence of mobile computing and ubiquitous computing. There were 130 papers submitted from 26 countries. For the first conference of this type the response was quite surprising. The audience is heavily attended by individuals from Asia.

There is a mix of academia and business. In both the keynote and panel discussions, the presenters confronted the issues of “economic reality.” Yet, when a technology is vision driven, few agree on how to make a business case. It appeared that many saw Bluetooth as the enabling wireless technology. Yet, one speaker from Europe stated that Ambient Intelligence does not imply that it must all be wireless. The presenter, Stefano Salsano, U. of Roma Tor Vergata, spoke of the IST – SIMPLICITY project (see the WAVE report on 13th IST Mobile & Wireless Communications Summit) and to him this project was not abstract but of how to make devices easy to use. Stefano addressed some of the practical issues of how to get service providers to let users own and carry ones personal state. This is getting close to some of the caching issues discussed by Professor Satyanarayanan. Bottom line, Mark Weiser stated in his classic paper "The Computer for the Twenty-First Century," Scientific American, pp. 94-10, September 1991 that it would take 20 years for this to be realized. 8 years remain to address many complex issues.

http://www.mobiquitous.org/