Cognitive computing for intelligent multimedia systems

Intelligent multimedia systems (IMMS) assist users in choosing the information content, and to design the information presentation, that satisfies their needs and idiosyncrasies. Towards this target, IMMSs posses and utilize knowledge concerning the means and function (purpose) of a presentation. ‘Means’ denote communication techniques (information carriers and modalities), as well as information content and information characteristics. The ‘function of a pr esentation’ denotes the communicative purpose of the presentation, methods for achieving that purpose, as well as a user's goals and tasks.

Download Full-text

Decimation of human face model for real-time animation in intelligent multimedia systems

Multimedia Tools and Applications ◽

10.1007/s11042-009-0411-8 ◽

2009 ◽

Vol 47 (1) ◽

pp. 147-162 ◽

Cited By ~ 3

Author(s):

Soo-Kyun Kim ◽

Syung-Og An ◽

Min Hong ◽

Doo-Soon Park ◽

Shin-Jin Kang

Keyword(s):

Real Time ◽

Multimedia Systems ◽

Human Face ◽

Face Model ◽

Intelligent Multimedia

Download Full-text

An Overview of Multimodal Interaction Techniques and Applications

Human Computer Interaction ◽

10.4018/978-1-87828-991-9.ch008 ◽

2009 ◽

pp. 95-101

Author(s):

Marie-Luce Bourguet

Keyword(s):

Computer Vision ◽

Speech Recognition ◽

Facial Expressions ◽

Multimodal Interaction ◽

Multimedia Systems ◽

Multimodal Interfaces ◽

Storage Devices ◽

Multimodal Systems ◽

Institute Of Technology ◽

Intelligent Multimedia

Desktop multimedia (multimedia personal computers) dates from the early 1970s. At that time, the enabling force behind multimedia was the emergence of the new digital technologies in the form of digital text, sound, animation, photography, and, more recently, video. Nowadays, multimedia systems mostly are concerned with the compression and transmission of data over networks, large capacity and miniaturized storage devices, and quality of services; however, what fundamentally characterizes a multimedia application is that it does not understand the data (sound, graphics, video, etc.) that it manipulates. In contrast, intelligent multimedia systems at the crossing of the artificial intelligence and multimedia disciplines gradually have gained the ability to understand, interpret, and generate data with respect to content. Multimodal interfaces are a class of intelligent multimedia systems that make use of multiple and natural means of communication (modalities), such as speech, handwriting, gestures, and gaze, to support human-machine interaction. More specifically, the term modality describes human perception on one of the three following perception channels: visual, auditive, and tactile. Multimodality qualifies interactions that comprise more than one modality on either the input (from the human to the machine) or the output (from the machine to the human) and the use of more than one device on either side (e.g., microphone, camera, display, keyboard, mouse, pen, track ball, data glove). Some of the technologies used for implementing multimodal interaction come from speech processing and computer vision; for example, speech recognition, gaze tracking, recognition of facial expressions and gestures, perception of sounds for localization purposes, lip movement analysis (to improve speech recognition), and integration of speech and gesture information. In 1980, the put-that-there system (Bolt, 1980) was developed at the Massachusetts Institute of Technology and was one of the first multimodal systems. In this system, users simultaneously could speak and point at a large-screen graphics display surface in order to manipulate simple shapes. In the 1990s, multimodal interfaces started to depart from the rather simple speech-and-point paradigm to integrate more powerful modalities such as pen gestures and handwriting input (Vo, 1996) or haptic output. Currently, multimodal interfaces have started to understand 3D hand gestures, body postures, and facial expressions (Ko, 2003), thanks to recent progress in computer vision techniques.

Download Full-text

An Overview of Multimodal Interaction Techniques and Applications

Intelligent Information Technologies ◽

10.4018/978-1-59904-941-0.ch001 ◽

2011 ◽

pp. 1-7

Author(s):

Marie-Luce Bourguet

Keyword(s):

Computer Vision ◽

Speech Recognition ◽

Facial Expressions ◽

Multimodal Interaction ◽

Multimedia Systems ◽

Multimodal Interfaces ◽

Storage Devices ◽

Multimodal Systems ◽

Institute Of Technology ◽

Intelligent Multimedia

Desktop multimedia (multimedia personal computers) dates from the early 1970s. At that time, the enabling force behind multimedia was the emergence of the new digital technologies in the form of digital text, sound, animation, photography, and, more recently, video. Nowadays, multimedia systems mostly are concerned with the compression and transmission of data over networks, large capacity and miniaturized storage devices, and quality of services; however, what fundamentally characterizes a multimedia application is that it does not understand the data (sound, graphics, video, etc.) that it manipulates. In contrast, intelligent multimedia systems at the crossing of the artificial intelligence and multimedia disciplines gradually have gained the ability to understand, interpret, and generate data with respect to content. Multimodal interfaces are a class of intelligent multimedia systems that make use of multiple and natural means of communication (modalities), such as speech, handwriting, gestures, and gaze, to support human-machine interaction. More specifically, the term modality describes human perception on one of the three following perception channels: visual, auditive, and tactile. Multimodality qualifies interactions that comprise more than one modality on either the input (from the human to the machine) or the output (from the machine to the human) and the use of more than one device on either side (e.g., microphone, camera, display, keyboard, mouse, pen, track ball, data glove). Some of the technologies used for implementing multimodal interaction come from speech processing and computer vision; for example, speech recognition, gaze tracking, recognition of facial expressions and gestures, perception of sounds for localization purposes, lip movement analysis (to improve speech recognition), and integration of speech and gesture information. In 1980, the put-that-there system (Bolt, 1980) was developed at the Massachusetts Institute of Technology and was one of the first multimodal systems. In this system, users simultaneously could speak and point at a large-screen graphics display surface in order to manipulate simple shapes. In the 1990s, multimodal interfaces started to depart from the rather simple speech-and-point paradigm to integrate more powerful modalities such as pen gestures and handwriting input (Vo, 1996) or haptic output. Currently, multimodal interfaces have started to understand 3D hand gestures, body postures, and facial expressions (Ko, 2003), thanks to recent progress in computer vision techniques.

Download Full-text