Tuyển tập báo cáo các nghiên cứu khoa học quốc tế ngành hóa học dành cho các bạn yêu hóa học tham khảo đề tài: The Catchment Feature Model: A Device for Multimodal Fusion and a Bridge between Signal and Sense | EURASIP Journal on Applied Signal Processing 2004 11 1619-1636 2004 Hindawi Publishing Corporation The Catchment Feature Model A Device for Multimodal Fusion and a Bridge between Signal and Sense Francis Quek Vision Interfaces and Systems Laboratory Center for Human Computer Interaction Virginia Polytechnic Institute and State University Blacksburg VA 24061 USA Email quek@ Received 24 October 2002 Revised 16 February 2004 The catchment feature model addresses two questions in the field of multimodal interaction how we bridge video and audio processing with the realities of human multimodal communication and how information from the different modes may be fused. We argue from a detailed literature review that gestural research has clustered around manipulative and semaphoric use of the hands motivate the catchment feature model psycholinguistic research and present the model. In contrast to whole gesture recognition the catchment feature model applies a feature decomposition approach that facilitates cross-modal fusion at the level of discourse planning and conceptualization. We present our experimental framework for catchment feature-based research cite three concrete examples of catchment features and propose new directions of multimodal research based on the model. Keywords and phrases multimodal interaction gesture interaction multimodal communications motion symmetries gesture space use. 1. INTRODUCTION The importance of gestures of hand head face eyebrows eye and body posture in human communication in conjunction with speech is self-evident. This paper advances a device known as the catchment 1 2 3 and the concept of a catchment feature that unifies what can reasonably be extracted from video imagery with human discourse. The catchment feature model also serves as the basis for multimodal fusion at this level of discourse conceptualization. This represents a new direction for gesture and speech analysis that makes each indispensable to the other. To .
