The Intelligent Classroom
Research: Plan Recognition
To effectively cooperate with the speaker, the Intelligent Classroom must act appropriately at the right moments. So, the Classroom must first understand what the speaker is doing and then carefully synchronize its actions with the speaker's. For example, when a speaker goes to the chalkboard to write, the Classroom should use two very different camera techniques: one for when he walks and the other for when he writes. If the Classroom uses the walking technique while the speaker is writing, people viewing the video feed won't be able to read his writing.
To address this challenge, the Classroom uses plan representations that explicitly represent the speaker's actions, the Classroom's actions, and how they should fit together. These plans are intended to represent a common understanding of how a speaker and an A/V assistant would interact. When the speaker is doing something, the Classroom monitors his progress through his part of the plan, waiting for the moments when the Classroom needs to act. For example, the "walk over to the chalkboard and write" plan has a process (sequence of actions) for the speaker's actions of moving to the board, stopping at it, beginning to write, and finishing. It also includes processes specifying how the Classroom should film the speaker and adjust the lights. Finally, the plan states that the camera technique should start changing as the speaker enters the chalkboard's vicinity.
The Classroom also uses these plan representations to reason about the speaker's actions at a higher level. While the speaker gives a presentation, the Classroom monitors the processes that serve as its understanding of the activity in the environment. These include processes for both the speaker's actions and the Classroom's actions (such as playing a video or showing a slide). When the Classroom observes the speaker taking an action (such as walking, gesturing or speaking), it tries to explain this action in the context of its understanding. That is, it looks through these processes for one that predicts that the speaker will perform that action.
However, if the Classroom doesn't find such a process, it must revise its understanding of what the speaker is doing: the speaker apparently isn't doing what the Classroom thought he was. The Classroom then hypothesizes new processes that explain the speaker's action. Initially, there may be several candidate explanations, but when the speaker's future actions contradict some of the proposed processes, they can be rejected, eventually leaving just the speaker's actual activity.
Maintained by franklin@cs.northwestern.edu
Last update: November 16th, 19101