Online Paper: Franklin-AAAI99-WS

Beyond "Next slide, please": The use of content and speech in multi-modal control

The Intelligent Classroom is an automated lecture facility where one of the primary goals is that speakers be able to control it by interacting with it as they would with a human A/V technician. In this paper we describe our research in imbedding Microsoft Powerpoint into the Intelligent Classroom. In particular we discuss how we use two modes of sensing (Computer Vision and Speech Recognition) to provide hands-free control of slide presentations. We look at utterances and gestures that serve as commands. And, more interestingly, we discuss some probabilistic techniques that can be used to match slide content to the speaker's words, providing new and useful ways of performing a presentation.