Đối với họ, văn bản, đồ họa, video và tất cả sẽ được các phương thức khác nhau trên màn hình máy tính, và ngôn ngữ nói một kiểu đặc biệt của phương thức mã hóa trên các phương tiện truyền thông âm thanh. | 174 Kristiina JOKINEN The ISLE NIMM standardization group for natural multimodal interaction Dybkjaer et al. 2002 also assumes a computer-oriented position but uses a two-way definition by conflating the code and modality. According to them medium is the physical channel for information encoding such as sounds movements etc. while modality is a particular way of encoding information in some medium. For them text graphics and video would all be different modalities on computer screen and spoken language a special type of modality encoded in audio media. We consider it important to distinguish code interaction language from modality and also be consistent with the human-oriented comprehension of modalities so that the term refers to different types of sensory information. We thus follow Maybury and Wahlster 1998 who offer the following definitions - Medium material on which or through which information is captured conveyed or interacted with . text audio video - Code system of symbols used for communication language gestures - Mode modality human perceptual systems that enable sensing vision auditory tactile olfaction taste . Graphics displayed on the computer screen is thus an instance of graphical output medium perceived through visual modality while speech uses audio medium microphone loudspeakers and auditory modality. Their definition of modality has also been criticized since it does not readily correspond to the way the term has been used in the literature on multimodal systems. In the strictest sense a system would need to process input that comes through two senses in order to be regarded as multimodal and thus . pen-based systems that use only pen would not be multimodal even though the input can be graphics and language since both of these are perceived visually. However the notion of code distinguishes these cases drawings and textual words apparently follow different symbolic interpretations and following the extended definition of a multimodal .
