CHIL computing to overcome techno-clutter

After building computers that paid no intention to communicating with humans, we have in recent years developed ever more sophisticated interfaces that put the "human in the loop" of computers. These interfaces have improved usability by providing more appealing output (graphics, animations), more easy to use input methods (mouse, pointing, clicking, dragging) and more natural interaction modes (speech, vision, gesture, etc.). Yet the productivity gains that have been promised have largely not been seen and human-machine interaction still remains a partially frustrating and tedious experience, full of techno-clutter and excessive attention required by the technical artifact.In this talk, I will argue, that we must transition to a third paradigm of computer use, in which we let people interact with people, and move the machine into the background to observe the humans' activities and to provide services implicitly, that is, -to the extent possible- without explicit request. Putting the "Computer in the Human Interaction Loop" (CHIL), instead of the other way round, however, brings formidable technical challenges. The machine must now always observe and understand humans, model their activities, their interaction with other humans, the human state as well as the state of the space they are in, and finally, infer intentions and needs. From a perceptual user interface point of view, we must process signals from sensors that are always on, frequently inappropriately positioned, and subject to much greater variablity. We must also not only recognize WHAT was seen or said in a given space, but also a broad range of additional information, such as the WHO, WHERE, HOW, TO WHOM, WHY, WHEN of human interaction and engagement.In this talk, I will describe a variety of multimodal interface technologies that we have developed to answer these questions and some preliminary CHIL type services that take advantage of such perceptual interfaces.