Adaptation and Personalization of Web-Based Multimedia Content

A traditional multimedia system presents the same static content and suggests the same next page to all users, even though they might have widely differing knowledge of the subject. Such a system suffers from an inability to be all things to all people, especially when the user population is relatively diverse. The rapid growth of mobile and wireless communication allowed service providers to develop new ways of interactions, enabling users to become accustomed to new means of multimedia-based service consumption in an anytime, anywhere, and anyhow manner. This chapter investigates the new multi-channel constraints and opportunities emerged by these technologies, as well as the new user-demanding requirements that arise. It further examines the relationship between the adaptation and personalization research considerations, and proposes a three-layer architecture for adaptation and personalization of Web-based multimedia content based on the “new” user profile, with visual, emotional, and cognitive processing parameters incorporated.

Expert-Novice Differences and Adaptive Multimedia

This chapter provides an overview of theoretical frameworks and empirical evidence for the design of adaptive multimedia that is tailored to individual levels of user expertise to optimize cognitive resources available for learning. Recent studies indicate that multimedia design principles that benefit low-knowledge users may disadvantage more experienced ones due to increase in cognitive load required for integration of presented information with available knowledge base. The major implication for multimedia design is the need to tailor instructional formats to individual levels of expertise. The suggested adaptive procedure is based on empirically-established interactions between levels of user proficiency and formats of multimedia presentations (the expertise reversal effect), and on real-time monitoring of users’ expertise using rapid cognitive diagnostic methods.

Perceptual Multimedia

In this chapter, we describe the results of empirical studies which examined the effect of cognitive style on the perceived quality of distributed multimedia. We use two dimensions of Cognitive Style Analysis, Field Dependent/Independent and Verbaliser/Visualiser, and the Quality of Perception metric to characterise the human perceptual experience. This is a metric which takes into account multimedia’s infotainment (combined informational and entertainment) nature, and comprises not only a human’s subjective level of enjoyment with regards to multimedia content quality, but also his/her ability to analyse, synthesise and assimilate the informational content of such presentations. Results show that multimedia content and dynamism are strong factors influencing perceptual quality.

Perceptual-Based Visualization of Auditory Information Using Visual Texture

This chapter investigates the use of visual texture for the visualization of multi-dimensional auditory information. Twenty subjects with a strong musical background performed a series of association tasks between high-level perceptual dimensions of visual texture and steady-state features of auditory timbre. The results indicated strong and intuitive mappings between (a) texture contrast and sharpness, (b) texture coarseness-granularity and compactness, and (c) texture periodicity and sensory dissonance. The findings contribute in setting the necessary groundwork for the application of empirically-derived auditory-visual mappings in multimedia environments.

Multimodel Expressions

In a culture which places increasing emphasis on happiness and wellbeing, multimedia technologies include emotional design to improve commercial edge. This chapter explores affective computing and illustrates how innovative technologies are capable of emotional recognition and display. Research in this domain has emphasised solving the technical difficulties involved, through the design of ever more complex recognition algorithms. But fundamental questions about the use of such technology remain neglected. Can it really improve human-computer interaction? For which types of application is it suitable? How is it best implemented? What ethical considerations are there? We review this field and discuss the need for user-centred design. We describe and give evidence from a study that explores some of the user issues in affective computing.

The Mathematics of Perception

This chapter summarizes the work on Mathematics of Perception performed by my research team between 2000 and 2005. To support personalization, a search engine must comprehend users’ query concepts (or perceptions), which are subjective and complicated to model. Traditionally, such query-concept comprehension has been performed through a process called “relevance feedback.” Our work formulates relevance feedback as a machine-learning problem when used with a small, biased training dataset. The problem arises because traditional machine learning algorithms cannot effectively learn a target concept when the training dataset is small and biased. My team has pioneered in developing a method of query-concept learning as the learning of a binary classifier to separate what a user wants from what she or he does not want, sorted out in a projected space. We have developed and published several algorithms to reduce data dimensions, to maximize the usefulness of selected training instances, to conduct learning on unbalanced datasets, to accurately account for perceptual similarity, to conduct indexing and learning in a non-metric, high-dimensional space, and to integrate perceptual features with keywords and contextual information. The technology of mathematics of perception encompasses an array of algorithms, and has been licensed by major companies for solving their image annotation, retrieval, and filtering problems.

Incorporating User Perception in Adaptive Video Streaming Systems

There is an increasing demand for streaming video applications over both the fixed Internet and wireless IP networks. The fluctuating bandwidth and time-varying delays of best-effort networks makes providing good quality streaming a challenge. Many adaptive video delivery mechanisms have been proposed over recent years; however, most do not explicitly consider user-perceived quality when making adaptations, nor do they define what quality is. This chapter describes research that proposes that an optimal adaptation trajectory through the set of possible encodings exists, and indicates how to adapt transmission in response to changes in network conditions in order to maximize user-perceived quality.

A Three-Layer Framework for ?QoS-Aware Service Design

While increasing bandwidth might be an acceptable way to maximize user utility, it is not always sufficient. This chapter presents a three-layer model of networked multimedia application based on a division into network, application, and usage. Quality measures for each layer are presented. Mental workload, task performance, and enjoyment are proposed as the most important variables at the usage layer. Interactions between the layers are described. Examples are given of the model which has been used in actual service design work. Finally, it is suggested that the model could also be used for dynamic adaptations.

Issues of Hand Preference in Computer Presented Information and Virtual Realities

This chapter presents the differences between left- and right-handed persons in the aspect of computer-presented information and virtual realities. It introduces five test scenarios and their results addressing this question. We showed that there are moderate differences between groups preferring different hands. The different needs of left- and right-handed people may play an important role in user-friendly interface and virtual environment design, since about a tenth of the population is left-handed. This could help to undo the difficulties that the left-handed and ambidextrous routinely encounter in their daily lives.

Using Real-Time Physiological Monitoring for Assessing Cognitive States

This chapter describes our research focused on deriving changing cognitive state information from the patterns of data acquired from the user, with the goal of using this information to improve the presentation of multimedia computer information. Detecting individual differences via performance and psychometric tools can be supplemented by using real-time physiological sensors. Described is an example computer task that demonstrates how cognitive load is manipulated. The different types of physiological and cognitive state measures are discussed along with their advantages and disadvantages. Experimental results from eye tracking and the pressures applied to a computer mouse are described in greater detail. Finally, adaptive information filtering is discussed as a model for using the physiological information to improve computer performance. Study results provide support that we can create effective ways to adapt to a person’s cognition in real time and thus facilitate real-world tasks.