Beyond synchronization

In the Western art music tradition, among many others, top ensembles are distinguished on the basis of their creative interpretation and expressivity, rather than purely on the precision of their synchronization. This chapter proposes that visual cues serve as social motivators during ensemble performance, promoting performers’ creative engagement with the music and each other. This chapter discusses findings from a study in which skilled duo musicians’ use of visual cues (eye gaze and body motion) was examined across the course of a rehearsal session. Results show that performers are driven to interact visually: (1) by temporal irregularity in the music and (2) by increased familiarity with the music and their co-performer. Synchronization success was unimpaired during a “blind” performance where performers could not see each other. Ensemble musicians thus choose to supplement their auditory interactions with visual cues despite their visual interactions offering no apparent benefit to synchronization.

Spatial Structure and Information Transfer in Visual Networks

In human and animal groups, social interactions often rely on the transmission of information via visual observation of the behavior of others. These visual interactions are governed by the laws of physics and sensory limits. Individuals appear smaller when far away and thus become harder to detect visually, while close by neighbors tend to occlude large areas of the visual field and block out interactions with individuals behind them. Here, we systematically study the effect of a group’s spatial structure, its density as well as polarization and aspect ratio of the physical bodies, on the properties of static visual interaction networks. In such a network individuals are connected if they can see each other as opposed to other interaction models such as metric or topological networks that omit these limitations due to the individual’s physical bodies. We find that structural parameters of the visual networks and especially their dependence on spatial group density are fundamentally different from the two other types. This results in characteristic deviations in information spreading which we study via the dynamics of two generic SIR-type models of social contagion on static visual and metric networks. We expect our work to have implications for the study of animal groups, where it could inform the study of functional benefits of different macroscopic states. It may also be applicable to the construction of robotic swarms communicating via vision or for understanding the spread of panics in human crowds.

Genetic mutation and biological pathway prediction based on whole slide images in breast carcinoma using deep learning

Breast carcinoma is the most common cancer among women worldwide that consists of a heterogeneous group of subtype diseases. The whole-slide images (WSIs) can capture the cell-level heterogeneity, and are routinely used for cancer diagnosis by pathologists. However, key driver genetic mutations related to targeted therapies are identified by genomic analysis like high-throughput molecular profiling. In this study, we develop a deep-learning model to predict the genetic mutations and biological pathway activities directly from WSIs. Our study offers unique insights into WSI visual interactions between mutation and its related pathway, enabling a head-to-head comparison to reinforce our major findings. Using the histopathology images from the Genomic Data Commons Database, our model can predict the point mutations of six important genes (AUC 0.68–0.85) and copy number alteration of another six genes (AUC 0.69–0.79). Additionally, the trained models can predict the activities of three out of ten canonical pathways (AUC 0.65–0.79). Next, we visualized the weight maps of tumor tiles in WSI to understand the decision-making process of deep-learning models via a self-attention mechanism. We further validated our models on liver and lung cancers that are related to metastatic breast cancer. Our results provide insights into the association between pathological image features, molecular outcomes, and targeted therapies for breast cancer patients.

First record of swimming speed of the Pacific sleeper shark Somniosus pacificus using a baited camera array

The Pacific sleeper shark Somniosus pacificus is one of the largest predators in deep Suruga Bay, Japan. A single individual of the sleeper shark (female, ~300 cm in total length) was observed with two baited camera systems deployed simultaneously on the deep seafloor in the bay. The first arrival was recorded 43 min after the deployment of camera #1 on 21 July 2016 at a depth of 609 m. The shark had several remarkable features, including the snout tangled in a broken fishing line, two torn anteriormost left-gill septums, and a parasitic copepod attached to each eye. The same individual appeared at camera #2, which was deployed at a depth of 603 m, ~37 min after it disappeared from camera #1 view. Finally, the same shark returned to camera #1 ~31 min after leaving camera #2. The distance between the two cameras was 436 m, and the average groundspeed and waterspeed of the shark were 0.21 and 0.25 m s−1, respectively, which were comparable with those of the Greenland shark Somniosus microcephalus (0.22–0.34 m s−1) exhibiting the slowest comparative swimming speed among fish species adjusted for size. The ambient water temperature of the Pacific sleeper shark was 5.3 °C, which is considerably higher than that of the Greenland shark (~2 °C). Such a low swimming speed might be explained by the ‘visual interactions hypothesis’, but it is not a consequence of the negative effects of cold water on their locomotor organs.

Directional effects of whole-body spinning and visual flow in virtual reality on vagal neuromodulation

BACKGROUND: Neural circuits allow whole-body yaw rotation to modulate vagal parasympathetic activity, which alters beat-to-beat variation in heart rate. The overall output of spinning direction, as well as vestibular-visual interactions on vagal activity still needs to be investigated. OBJECTIVE: This study investigated direction-dependent effects of visual and natural vestibular stimulation on two autonomic responses: heart rate variability (HRV) and pupil diameter. METHODS: Healthy human male subjects (n = 27) underwent constant whole-body yaw rotation with eyes open and closed in the clockwise (CW) and anticlockwise (ACW) directions, at 90°/s for two minutes. Subjects also viewed the same spinning environments on video in a VR headset. RESULTS: CW spinning significantly decreased parasympathetic vagal activity in all conditions (CW open p = 0.0048, CW closed p = 0.0151, CW VR p = 0.0019,), but not ACW spinning (ACW open p = 0.2068, ACW closed p = 0.7755, ACW VR p = 0.1775,) as indicated by an HRV metric, the root mean square of successive RR interval differences (RMSSD). There were no direction-dependent effects of constant spinning on sympathetic activity inferred through the HRV metrics, stress index (SI), sympathetic nervous system index (SNS index) and pupil diameter. Neuroplasticity in the CW eyes closed and CW VR conditions post stimulation was observed. CONCLUSIONS: Only one direction of yaw spinning, and visual flow caused vagal nerve neuromodulation and neuroplasticity, resulting in an inhibition of parasympathetic activity on the heart, to the same extent in either vestibular or visual stimulation. These results indicate that visual flow in VR can be used as a non-electrical method for vagus nerve inhibition without the need for body motion in the treatment of disorders with vagal overactivity. The findings are also important for VR and spinning chair based autonomic nervous system modulation protocols, and the effects of motion integrated VR.

Cortical response to naturalistic stimuli is largely predictable with deep neural networks

Naturalistic stimuli, such as movies, activate a substantial portion of the human brain, invoking a response shared across individuals. Encoding models that predict neural responses to arbitrary stimuli can be very useful for studying brain function. However, existing models focus on limited aspects of naturalistic stimuli, ignoring the dynamic interactions of modalities in this inherently context-rich paradigm. Using movie-watching data from the Human Connectome Project, we build group-level models of neural activity that incorporate several inductive biases about neural information processing, including hierarchical processing, temporal assimilation, and auditory-visual interactions. We demonstrate how incorporating these biases leads to remarkable prediction performance across large areas of the cortex, beyond the sensory-specific cortices into multisensory sites and frontal cortex. Furthermore, we illustrate that encoding models learn high-level concepts that generalize to task-bound paradigms. Together, our findings underscore the potential of encoding models as powerful tools for studying brain function in ecologically valid conditions.

The sensory impacts of climate change: bathymetric shifts and visually mediated interactions in aquatic species

Visual perception is, in part, a function of the ambient illumination spectrum. In aquatic environments, illumination depends upon the water's optical properties and depth, both of which can change due to anthropogenic impacts: turbidity is increasing in many aquatic habitats, and many species have shifted deeper in response to warming surface waters (known as bathymetric shifts). Although increasing turbidity and bathymetric shifts can result in similarly large changes to a species' optical environment, no studies have yet examined the impact of the latter on visually mediated interactions. Here, we examine a potential link between climate change and visual perception, with a focus on colour. We discuss (i) what is known about bathymetric shifts; (ii) how the impacts of bathymetric shifts on visual interactions may be distributed across species; (iii) which interactions might be affected; and (iv) the ways that animals have to respond to these changes. As warming continues and temperature fluctuations grow more extreme, many species may move into even deeper waters. There is thus a need for studies that examine how such shifts can affect an organism's visual world, interfere with behaviour, and impact fitness, population dynamics, and community structure.

LITERARY CARTOGRAPHY: MAP AS A PARATEXTUAL ELEMENT IN BRITISH CHILDREN’S LITERATURE

Объект исследования данной статьи – литературная карта как визуальный элемент детской книги на рубеже XIX–XX веков – малоизученная область визуальной культуры детства. Карта становится важной составляющей визуальной культуры детства во второй половине XIX века. Целью исследования было проследить становление картографической традиции в детской литературе Великобритании. Для достижения поставленной цели потребовалось, во-первых, изучить ведущие современные направления в изучении литературной карты. Во-вторых, сформировать терминологический аппарат для описания карты как документального и эстетического объекта в рамках литературного произведения, который в силу малой изученности отсутствует в русском языке. В-третьих, описать пять литературных карт из классических произведений британской детской литературы Золотого периода в их взаимодействии с текстом. В результате работы автор приходит к выводу, что, являясь важным паратекстуальным элементом детской книги, эти карты представляют различные типы взаимодействия с основным текстом: карта-сюжет, карта-документ эпохи, карта-рассказчик, карта-память. Since one of the first representations of the Earth in “The Map Psalter”, marine maps from the Age of Discovery and the first literary atlases, maps have held a special place in British culture. Since the map of the Treasure Island, which is considered to be a pioneer of the kind, from Robert Louis Stevenson’s novel of the same name, maps have always played a significant role in British children’s literature. A literary map, especially a map in children’s books, is an important paratextual element. Although the roles and functions of maps may vary greatly, the place of a map (most frequently it is an endpaper or a frontispiece) makes literary cartography the first visual element for the reader, which enables a map to set the setting, genre, and particular audience expectations. The fact that it is not an obligatory element of a book makes the presence of a map in a book an essential part of the author’s artistic vision and a key (para)textual element of the book. The five maps from the classic books written for younger readers between 1883 and 1926 may prove that maps perform multiple functions and play a greater role than that of beautiful drawings on frontispieces. The maps are the 17th-century marine map of the imaginary island from Robert Louis Stevenson’s Treasure Island; the actual map of India from Rudyard Kipling’s Kim; the map of Kensington Gardens presumably drawn by a child from James M. Barrie’s Peter Pan in Kensington Gardens; the map of the Thames Valley inhibited by anthropomorphic animals from Kenneth Graham’s The Wind in the Willows. The analysis of these maps’ paratextual powers and textual-visual interactions leads to the conclusion that the five literary maps from the classic children’s books of the Golden Age period reveal the five potential ways of interaction between the textual and the visual: map as a plot device, map as a document, map as a narrator, map as the transcendent, and map as memory, correspondingly. The conclusion poses the following questions: What happens to maps during the act of translation from English into Russian or any other language? Is it possible to translate cartography? How crucial is the omission of a map? The answers to these questions are yet to be discovered.