Artistic Style Meets Artificial Intelligence

Recent developments in neural network image processing motivate the question, how these technologies might better serve visual artists. Research goals to date have largely focused on either pastiche interpretations of what is framed as artistic “style” or seek to divulge heretofore unimaginable dimensions of algorithmic “latent space,” but have failed to address the process an artist might actually pursue, when engaged in the reflective act of developing an image from imagination and lived experience. The tools, in other words, are constituted in research demonstrations rather than as tools of creative expression. In this article, the authors explore the phenomenology of the creative environment afforded by artificially intelligent image transformation and generation, drawn from autoethnographic reviews of the authors’ individual approaches to artificial intelligence (AI) art. They offer a post-phenomenology of “neural media” such that visual artists may begin to work with AI technologies in ways that support naturalistic processes of thinking about and interacting with computationally mediated interactive creation.

Artistic Style Meets Artificial Intelligence

Recent developments in neural network image processing motivate the question, how these technologies might better serve visual artists. Research goals to date have largely focused on either pastiche interpretations of what is framed as artistic “style” or seek to divulge heretofore unimaginable dimensions of algorithmic “latent space,” but have failed to address the process an artist might actually pursue, when engaged in the reflective act of developing an image from imagination and lived experience. The tools, in other words, are constituted in research demonstrations rather than as tools of creative expression. In this article, the authors explore the phenomenology of the creative environment afforded by artificially intelligent image transformation and generation, drawn from autoethnographic reviews of the authors’ individual approaches to artificial intelligence (AI) art. They offer a post-phenomenology of “neural media” such that visual artists may begin to work with AI technologies in ways that support naturalistic processes of thinking about and interacting with computationally mediated interactive creation.

Introducing CatchU TM : A Novel Multisensory Tool for Assessing Patients’ Risk of Falling †

To date, only a few studies have investigated the clinical translational value of multisensory integration. Our previous research has linked the magnitude of visual-somatosensory integration (measured behaviorally using simple reaction time tasks) to important cognitive (attention) and motor (balance, gait, and falls) outcomes in healthy older adults. While multisensory integration effects have been measured across a wide array of populations using various sensory combinations and different neuroscience research approaches, multisensory integration tests have not been systematically implemented in clinical settings. We recently developed a step-by-step protocol for administering and calculating multisensory integration effects to facilitate innovative and novel translational research across diverse clinical populations and age-ranges. In recognizing that patients with severe medical conditions and/or mobility limitations often experience difficulty traveling to research facilities or joining time-demanding research protocols, we deemed it necessary for patients to be able to benefit from multisensory testing. Using an established protocol and methodology, we developed a multisensory falls-screening tool called CatchU TM (an iPhone app) to quantify multisensory integration performance in clinical practice that is currently undergoing validation studies. Our goal is to facilitate the identification of patients who are at increased risk of falls and promote physician-initiated falls counseling during clinical visits (e.g., annual wellness, sick, or follow-up visits). This will thereby raise falls-awareness and foster physician efforts to alleviate disability, promote independence, and increase quality of life for our older adults. This conceptual overview highlights the potential of multisensory integration in predicting clinical outcomes from a research perspective, while also showcasing the practical application of a multisensory screening tool in routine clinical practice.

Enhanced Peripheral Face Processing in Deaf Individuals †

Studies of compensatory changes in visual functions in response to auditory loss have shown that enhancements tend to be restricted to the processing of specific visual features, such as motion in the periphery. Previous studies have also shown that deaf individuals can show greater face processing abilities in the central visual field. Enhancements in the processing of peripheral stimuli are thought to arise from a lack of auditory input and subsequent increase in the allocation of attentional resources to peripheral locations, while enhancements in face processing abilities are thought to be driven by experience with American sign language and not necessarily hearing loss. This combined with the fact that face processing abilities typically decline with eccentricity suggests that face processing enhancements may not extend to the periphery for deaf individuals. Using a face matching task, the authors examined whether deaf individuals’ enhanced ability to discriminate between faces extends to the peripheral visual field. Deaf participants were more accurate than hearing participants in discriminating faces presented both centrally and in the periphery. Their results support earlier findings that deaf individuals possess enhanced face discrimination abilities in the central visual field and further extend them by showing that these enhancements also occur in the periphery for more complex stimuli.

Low-contrast Acuity Under Strong Luminance Dynamics and Potential Benefits of Divisive Display Augmented Reality

Understanding and predicting outdoor visual performance in augmented reality (AR) requires characterizing and modeling vision under strong luminance dynamics, including luminance differences of 10000-to-1 in a single image (high dynamic range, HDR). Classic models of vision, based on displays with 100-to-1 luminance contrast, have limited ability to generalize to HDR environments. An important question is whether low-contrast visibility, potentially useful for titrating saliency for AR applications, is resilient to saccade-induced strong luminance dynamics. The authors developed an HDR display system with up to 100,000-to-1 contrast and assessed how strong luminance dynamics affect low-contrast visual acuity. They show that, immediately following flashes of 25× or 100× luminance, visual acuity is unaffected at 90% letter Weber contrast and only minimally affected at lower letter contrasts (up to +0.20 LogMAR for 10% contrast). The resilience of low-contrast acuity across luminance changes opens up research on divisive display AR (ddAR) to effectively titrate salience under naturalistic HDR luminance.

Adaptive Camouflage for Moving Objects

Targets that are well camouflaged under static conditions are often easily detected as soon as they start moving. We investigated and evaluated ways to design camouflage that dynamically adapts to the background and conceals the target while taking the variation in potential viewing directions into account. In a human observer experiment, recorded imagery was used to simulate moving (either walking or running) and static soldiers, equipped with different types of camouflage patterns and viewed from different directions. Participants were instructed to detect the soldier and to make a rapid response as soon as they have identified the soldier. Mean target detection rate was compared between soldiers in standard (Netherlands) Woodland uniform, in static camouflage (adapted to the local background) and in dynamically adapting camouflage. We investigated the effects of background type and variability on detection performance by varying the soldiers’ environment (such as bushland and urban). In general, detection was easier for dynamic soldiers compared to static soldiers, confirming that motion breaks camouflage. Interestingly, we show that motion onset and not motion itself is an important feature for capturing attention. Furthermore, camouflage performance of the static adaptive pattern was generally much better than for the standard Woodland pattern. Also, camouflage performance was found to be dependent on the background and the local structures around the soldier. Interestingly, our dynamic camouflage design outperformed a method which simply displays the ‘exact’ background on the camouflage suit (as if it was transparent), since it is better capable of taking the variability in viewing directions into account. By combining new adaptive camouflage technologies with dynamic adaptive camouflage designs such as the one presented here, it may become feasible to prevent detection of moving targets in the (near) future.

Perception and Appreciation of Tactile Objects: The Role of Visual Experience and Texture Parameters †

This exploratory study was designed to examine the effects of visual experience and specific texture parameters on both discriminative and aesthetic aspects of tactile perception. To this end, the authors conducted two experiments using a novel behavioral (ranking) approach in blind and (blindfolded) sighted individuals. Groups of congenitally blind, late blind, and (blindfolded) sighted participants made relative stimulus preference, aesthetic appreciation, and smoothness or softness judgment of two-dimensional (2D) or three-dimensional (3D) tactile surfaces through active touch. In both experiments, the aesthetic judgmen t was assessed on three affective dimensions, Relaxation, Hedonics, and Arousal, hypothesized to underlie visual aesthetics in a prior study. Results demonstrated that none of these behavioral judgments significantly varied as a function of visual experience in either experiment. However, irrespective of visual experience, significant differences were identified in all these behavioral judgments across the physical levels of smoothness or softness. In general, 2D smoothness or 3D softness discrimination was proportional to the level of physical smoothness or softness. Second, the smoother or softer tactile stimuli were preferred over the rougher or harder tactile stimuli. Third, the 3D affective structure of visual aesthetics appeared to be amodal and applicable to tactile aesthetics. However, analysis of the aesthetic profile across the affective dimensions revealed some striking differences between the forms of appreciation of smoothness and softness, uncovering unanticipated substructures in the nascent field of tactile aesthetics. While the physically softer 3D stimuli received higher ranks on all three affective dimensions, the physically smoother 2D stimuli received higher ranks on the Relaxation and Hedonics but lower ranks on the Arousal dimension. Moreover, the Relaxation and Hedonics ranks accurately overlapped with one another across all the physical levels of softness/hardness, but not across the physical levels of smoothness/roughness. These findings suggest that physical texture parameters not only affect basic tactile discrimination but differentially mediate tactile preferences, and aesthetic appreciation. The theoretical and practical implications of these novel findings are discussed.

Smelling Sensations: Olfactory Crossmodal Correspondences †

Olfaction is ingrained into the fabric of our daily lives and constitutes an integral part of our perceptual reality. Within this reality, there are crossmodal interactions and sensory expectations; understanding how olfaction interacts with other sensory modalities is crucial for augmenting interactive experiences with more advanced multisensorial capabilities. This knowledge will eventually lead to better designs, more engaging experiences, and enhancing the perceived quality of experience. Toward this end, the authors investigated a range of crossmodal correspondences between ten olfactory stimuli and different modalities (angularity of shapes, smoothness of texture, pleasantness, pitch, colors, musical genres, and emotional dimensions) using a sample of 68 observers. Consistent crossmodal correspondences were obtained in all cases, including our novel modality (the smoothness of texture). These associations are most likely mediated by both the knowledge of an odor’s identity and the underlying hedonic ratings: the knowledge of an odor’s identity plays a role when judging the emotional and musical dimensions but not for the angularity of shapes, smoothness of texture, perceived pleasantness, or pitch. Overall, hedonics was the most dominant mediator of crossmodal correspondences.

FP-Nets for Blind Image Quality Assessment

Feature-Product networks (FP-nets) are a novel deep-network architecture inspired by principles of biological vision. These networks contain the so-called FP-blocks that learn two different filters for each input feature map, the outputs of which are then multiplied. Such an architecture is inspired by models of end-stopped neurons, which are common in cortical areas V1 and especially in V2. The authors here use FP-nets on three image quality assessment (IQA) benchmarks for blind IQA. They show that by using FP-nets, they can obtain networks that deliver state-of-the-art performance while being significantly more compact than competing models. A further improvement that they obtain is due to a simple attention mechanism. The good results that they report may be related to the fact that they employ bio-inspired design principles.