Visual speech is processed differently in auditory and visual cortex: evidence from MEG and partial coherence analysis

Speech perception in noisy environments is enhanced by seeing facial movements of communication partners. However, the neural mechanisms by which audio and visual speech are combined are not fully understood. We explore MEG phase locking to auditory and visual signals in MEG recordings from 14 human participants (6 female) that reported words from single spoken sentences. We manipulated the acoustic clarity and visual speech signals such that critical speech information is present in auditory, visual or both modalities. MEG coherence analysis revealed that both auditory and visual speech envelopes (auditory amplitude modulations and lip aperture changes) were phase-locked to 2-6Hz brain responses in auditory and visual cortex, consistent with entrainment to syllable-rate components. Partial coherence analysis was used to separate neural responses to correlated audio-visual signals and showed non-zero phase locking to auditory envelope in occipital cortex during audio-visual (AV) speech. Furthermore, phase-locking to auditory signals in visual cortex was enhanced for AV speech compared to audio-only (AO) speech that was matched for intelligibility. Conversely, auditory regions of the superior temporal gyrus (STG) did not show above-chance partial coherence with visual speech signals during AV conditions, but did show partial coherence in VO conditions. Hence, visual speech enabled stronger phase locking to auditory signals in visual areas, whereas phase-locking of visual speech in auditory regions only occurred during silent lip-reading. Differences in these cross-modal interactions between auditory and visual speech signals are interpreted in line with cross-modal predictive mechanisms during speech perception.

Environmental thermal influence over soundscape perception: a test room experimental campaign involving the psychological and physiological description of the indoor environment

Human environmental perception leads occupants’ behaviour when interacting with buildings components, affecting the final building energy performance. A solid understanding of human comfort perception includes simultaneous multisensory stimuli and cross-modal interactions among different comfort domains. This study aims to explore the cross-modal effect between thermal and acoustic domains. Each of the 40 subjects took part in a multisensory survey under two different stationary environmental temperature settings. Results show that people in thermally warm conditions are less confident in describing the provided acoustic records. To perform the same procedure but providing a decreasing air temperature ramp would lead to a better interpretation of the results of this campaign.

An Initial Agent-Based Model for E-scooter Accommodation into Existing Urban Streetscapes

New forms of mobility reshape the transportation landscape, changing movement for both their users and others in the environment. The transition period during which novel forms of travel are being explored can be a challenging time while the use of spaces must be renegotiated. E-scooters, which have recently been more widely introduced to the UK, are experiencing such a moment as riders, planners, and other users of the streetscape are determining what role this technology will play in communities. The data gaps surrounding e-scooters can make this an especially difficult question for planners because of the cost of gathering relevant observational data, much of which is held under private company ownership. In light of this, this work presents an agent-based model developed to examine the integration of e-scooters into existing streetscapes. Agent-based models explore phenomena through focusing on individual behaviour and rules which in turn gives rise to emergent large scale patterns. These patterns can be dissected and interrogated with a variety of tools, allowing us to tease out individual as well as group experiences of different scenarios. An agent-based approach allows us to capture the individual behaviours of e-scooter users and those of cyclists, drivers of variously sized vehicles, pedestrians, and others present in the environment. By focusing on the interactions of these various street users, we can explore how different approaches to e-scooter integration may fare relative to varying street configurations. Their decision frameworks are informed by observational studies of e-scooter users in order to augment the available data. We discuss the current state of understanding e-scooter behaviour and the potential modelling applications, present an initial behavioural framework of e-scooter decision making and inter-modal interactions, and highlight some preliminary results examining the differences between e-scooters operating on roads versus shared segregated cycle lanes. The work concludes with a case study comparing two modelled scenarios, one including a segregated cycle lane and one without. Drawing upon metrics such as the route segmentation/ cut-off rate and average travel comfort, we can more precisely explore how new forms of mobility will influence different kinds of street users in order to better understand the trade-offs associated with different paths forward.

Optimization of Multimodal Discrete Network Design Problems Based on Super Networks

In this paper, we investigate the multimodal discrete network design problem that simultaneously optimizes the car, bus, and rail transit network, in which inter-modal transfers are achieved by slow traffic modes including walking and bike-sharing. Specifically, a super network topology is presented to signify the modal interactions. Then, the generalized cost formulas of each type of links in the super network are defined. And based on the above formulas a bi-objective programming model is proposed to minimize the network operation cost and construction cost with traffic flow equilibrium constraints, investment constraints and expansion constraints. Moreover, a hybrid heuristic algorithm that combines the minimum cost flow algorithm and simulated annealing algorithm is presented to solve the proposed model. Finally, the effectiveness of the proposed model and algorithm is evaluated through two numerical tests: a simple test network and an actual multimodal transport network.

Odor–Taste–Texture Interactions as a Promising Strategy to Tackle Adolescent Overweight

The adolescence period is characterized by a considerable risk to weight gain due to the high consumption of food rich in sugar. A promising strategy to reduce sugar consumption may lie in exploiting the ability of our senses to interact to each other (cross-modal interactions). The aims were to investigate the cross-modal interactions and gustatory function in normal-weight and overweight adolescents. Fifty adolescents (25 overweight and 25 normal-weight) were involved. Subjects rated liking and attribute intensity in pudding samples obtained by adding vanilla aroma (0.1%; 0.3%), butter aroma (0.05%; 0.1%) or a thickener agent (1%; 1.5%) to a base formulation. The gustatory function was also measured through the “taste strips” methodology. Overweight adolescents were found to have a significantly (p < 0.001) worse ability to correctly identify all tastes. Cross-modal interactions occurred differently according to their body mass index, with a significant increase (p < 0.05) in sensory desirable characteristics (e.g., sweet and creaminess) due to aroma addition, especially in overweight subjects. Furthermore, butter aroma significantly increased hedonic responses only in overweight subjects. Tricking our senses in the way of perceiving sensory attributes could be a promising strategy to develop innovative food formulations with a reduced sugar amount, which will lead to a potential decrease in caloric intake and help to tackle the obesity epidemic.

Mobility-as-a-Service: Simulation of Multi-Modal Operations in Low-Density Cities

In the case of the low-density city, empirical evidence continuously demonstrates that transit investment is not a magic bullet. Desirable outcomes are not guaranteed and are often dependent on development density and other urban characteristics. Mobility-as-a-service (MaaS) presents a new approach: a digital platform providing access to multi-modal travel alternatives and totally comprehensive integrated trip-making, planning, and payment services. Review of the literature highlights shortcomings in traditional transportation planning by examining aspects of multi-modal planning such as adoption, parterships, operations, integration, capacity implications, and impact analyses. To enhance the practice of multi-modal planning, the following experiment evaluates various performance measures and inter-modal interactions on International Drive in Orlando, Florida, U.S., via D- and I-optimal experimental designs in a simulated MaaS network. Alternative scenarios are developed comparing varied modal shares across five travel modes: personal vehicles, transit, ridesourcing (or ride-hailing), micro-mobility, and walking. The modal effects are analyzed to highlight the strengths and weakness of each mode under a variety of congestion conditions. While transit enjoys the lowest impact per person, ridesourcing demonstrates adverse effects across all measures. Based on the novel interactions of transit and ridesourcing with directional demand, strategies are outlined for optimizing ridesourcing-transit integration to reduce route travel time, queuing, and overall network delay. The performance impacts of curbside facilities are also discussed for improved multi-modal integration at the street level. These findings are applied to propose a framework for effective planning and implementation of mobility services in low-density cities, focused on operations, city-level connectivity, and curbside management.