Cross-Modality Transfer Learning for Image-Text Information Management

In the past decades, information from all kinds of data has been on a rapid increase. With state-of-the-art performance, machine learning algorithms have been beneficial for information management. However, insufficient supervised training data is still an adversity in many real-world applications. Therefore, transfer learning (TF) was proposed to address this issue. This article studies a not well investigated but important TL problem termed cross-modality transfer learning (CMTL). This topic is closely related to distant domain transfer learning (DDTL) and negative transfer. In general, conventional TL disciplines assume that the source domain and the target domain are in the same modality. DDTL aims to make efficient transfers even when the domains or the tasks are entirely different. As an extension of DDTL, CMTL aims to make efficient transfers between two different data modalities, such as from image to text. As the main focus of this study, we aim to improve the performance of image classification by transferring knowledge from text data. Previously, a few CMTL algorithms were proposed to deal with image classification problems. However, most existing algorithms are very task specific, and they are unstable on convergence. There are four main contributions in this study. First, we propose a novel heterogeneous CMTL algorithm, which requires only a tiny set of unlabeled target data and labeled source data with associate text tags. Second, we introduce a latent semantic information extraction method to connect the information learned from the image data and the text data. Third, the proposed method can effectively handle the information transfer across different modalities (text-image). Fourth, we examined our algorithm on a public dataset, Office-31. It has achieved up to 5% higher classification accuracy than “non-transfer” algorithms and up to 9% higher than existing CMTL algorithms.

Heterosynaptic NMDA Receptor Plasticity in Hippocampal Dentate Granule Cells

The dentate gyrus is a key relay station that controls information transfer from the entorhinal cortex to the hippocampus proper. This process heavily relies on dendritic integration by dentate granule cells (GCs) of excitatory synaptic inputs from medial and lateral entorhinal cortex via medial and lateral perforant paths (MPP and LPP, respectively). N-methyl-D-aspartate receptors (NMDARs) can contribute significantly to the integrative properties of neurons. While early studies reported that excitatory inputs from entorhinal cortex onto GCs can undergo activity-dependent long-term plasticity of NMDAR-mediated transmission, the input-specificity of this plasticity along the dendritic axis remains unknown. Here, we examined the NMDAR plasticity rules at MPP-GC and LPP-GC synapses using physiologically relevant patterns of stimulation in acute rat hippocampal slices. We found that MPP-GC, but not LPP-GC synapses, expressed homosynaptic NMDAR-LTP. In addition, induction of NMDAR-LTP at MPP-GC synapses heterosynaptically potentiated distal LPP-GC NMDAR plasticity. The same stimulation protocol induced homosynaptic α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid receptor (AMPAR)-LTP at MPP-GC but heterosynaptic AMPAR-LTD at distal LPP synapses, demonstrating that NMDAR and AMPAR are governed by different plasticity rules. Remarkably, heterosynaptic but not homosynaptic NMDAR-LTP required Ca2+ release from intracellular, ryanodine-dependent Ca2+ stores. Lastly, the induction and maintenance of both homo- and heterosynaptic NMDAR-LTP were blocked by GluN2D antagonism, suggesting the recruitment of GluN2D-containing receptors to the synapse. Our findings uncover a mechanism by which distinct inputs to the dentate gyrus may interact functionally and contribute to hippocampal-dependent memory formation.

How many bits of information can be transferred between residues in a protein and how fast?

Time resolved Raman and infrared spectroscopy experiments show the basic features of information transfer between residues in proteins. Here, we present the theoretical basis of information transfer using a simple elastic net model and recently developed entropy transfer concept in proteins. Mutual information between two residues is a measure of communication in proteins which shows the maximum amount of information that may be transferred between two residues. However, it does not explain the actual amount of transfer nor the transfer rate of information between residues. For this, dynamic equations of the system are needed. We used the Schreiber theory of information transfer and the Gaussian network Model of proteins, together with the solution of the Langevin equation, to quantify allosteric information transfer. Results of the model are in perfect agreement with ultraviolet resonance Raman measurements. Analysis of the allosteric protein Human NAD-dependent isocitrate dehydrogenase shows that a multitude of paths contribute collectively to information transfer. While the peak values of information transferred are small relative to information content of residues, considering the estimated transfer rates, which are in the order of megabits per second, sustained transfer during the activity time-span of proteins may be significant.

Visualization of Speech Perception Analysis via Phoneme Alignment: A Pilot Study

Objective: Speech tests assess the ability of people with hearing loss to comprehend speech with a hearing aid or cochlear implant. The tests are usually at the word or sentence level. However, few tests analyze errors at the phoneme level. So, there is a need for an automated program to visualize in real time the accuracy of phonemes in these tests.Method: The program reads in stimulus-response pairs and obtains their phonemic representations from an open-source digital pronouncing dictionary. The stimulus phonemes are aligned with the response phonemes via a modification of the Levenshtein Minimum Edit Distance algorithm. Alignment is achieved via dynamic programming with modified costs based on phonological features for insertion, deletions and substitutions. The accuracy for each phoneme is based on the F1-score. Accuracy is visualized with respect to place and manner (consonants) or height (vowels). Confusion matrices for the phonemes are used in an information transfer analysis of ten phonological features. A histogram of the information transfer for the features over a frequency-like range is presented as a phonemegram.Results: The program was applied to two datasets. One consisted of test data at the sentence and word levels. Stimulus-response sentence pairs from six volunteers with different degrees of hearing loss and modes of amplification were analyzed. Four volunteers listened to sentences from a mobile auditory training app while two listened to sentences from a clinical speech test. Stimulus-response word pairs from three lists were also analyzed. The other dataset consisted of published stimulus-response pairs from experiments of 31 participants with cochlear implants listening to 400 Basic English Lexicon sentences via different talkers at four different SNR levels. In all cases, visualization was obtained in real time. Analysis of 12,400 actual and random pairs showed that the program was robust to the nature of the pairs.Conclusion: It is possible to automate the alignment of phonemes extracted from stimulus-response pairs from speech tests in real time. The alignment then makes it possible to visualize the accuracy of responses via phonological features in two ways. Such visualization of phoneme alignment and accuracy could aid clinicians and scientists.

Virtual farm tours—Virtual reality glasses and tablets are suitable tools to provide insights into pig husbandry

Apart from improving husbandry conditions and animal welfare, there is a clear public demand to increase transparency in agricultural activities. Personal farm tours have shown to be appreciated by citizens but are limited in their impact because of hygiene requirements and accessibility. Virtual farm tours are a promising approach to overcome these limitations but evidence on their perceptions is missing. This study analyzes how a virtual farm tour is perceived by showing participants (n = 17) a 360-degree video of a conventional pig fattening pen on a tablet and via virtual reality (VR) glasses. Semi-structured in-depth interviews were conducted to analyze perceptions and level of immersion and to elicit differences between media devices. Participants’ perception of the pig fattening pen was rather poor and depended on the recording perspective as well as on the media device. However, housing conditions were perceived more positively compared to the image participants had in mind prior to the study, and thus the stable was considered as a rather positive example. Participants described virtual farm tours as suitable tool to improve transparency and information transfer and to gain insights into husbandry conditions. They appreciated the comfortable and entertaining character of both media devices and named various possibilities for implementation. However, VR glasses were favored regarding the higher realistic and entertaining value, while the tablet was considered beneficial in terms of usability. The presentation of video sequences without additional explanations about the farm or the housing conditions were claimed insufficient to give an adequate understanding of the seen content.

Potential uses of AI for perioperative nursing handoffs: a qualitative study.

Objective: Situational awareness and anticipatory guidance for nurses receiving a patient after surgery are key to patient safety. Little work has defined the role of artificial intelligence (AI) to support these functions during nursing handoff communication or patient assessment. We used interviews and direct observations to better understand how AI could work in this context. Materials and Methods: 58 handoffs were observed of patients entering and leaving the post-anesthesia care unit at a single center. 11 nurses participated in semi-structured interviews. Mixed inductive-deductive thematic analysis extracted major themes and subthemes around roles for AI supporting postoperative nursing. Results: Four themes emerged from the interviews: (1) Nurse understanding of patient condition guides care decisions, (2) Handoffs are important to nurse situational awareness; problem focus and information transfer may be improved by AI, (3) AI may augment nurse care decision making and team communication, (4) User experience and information overload are likely barriers to using AI. Key subthemes included that AI-identified problems would be discussed at handoff and team communications, that AI-estimated elevated risks would trigger patient re-evaluation, and that AI-identified important data may be a valuable addition to nursing assessment. Discussion and Conclusion: Most research on postoperative handoff communication relies on structured checklists. Our results suggest that properly designed AI tools might facilitate postoperative handoff communication for nurses by identifying elevated risks faced by a specific patient, triggering discussion on those topics.

3D Printed Polymer Photonic Topological Insulators and their Robustness to Fabrication Disorder

Photonic topological insulators have emerged as an exciting new platform for backscatter-free waveguiding even in the presence of defects, with applications in robust long-range energy and quantum information transfer, spectroscopy and sensing, chiral quantum optics, and optoelectronics. We demonstrate a design for spin-Hall photonic topological insulators with remarkably low refractive index contrast, enabling the synthesis of photonic topological waveguides from polymeric materials for the first time. Our design is compatible with additive manufacturing methods, including fused filament fabrication for microwave frequencies, and constitutes the first demonstration of a 3D printed all-dielectric photonic topological insulator. We combine rapid device fabrication through 3D printing with high-speed FDTD simulation to quantify topological protection of transmission through “omega” shaped bent topological waveguides and find that one corner in the waveguide is 3-5 times more robust to disorder than the other. This dichotomy, a new empirical design rule for ℤ2 topological insulator devices, is shown to originate in the fundamental system symmetries and is illustrated via the distributions of Poynting vectors that describe energy flow through the waveguide. Taken together, our demonstration of 3D printed polymeric spin-Hall photonic topological insulators paired with quantification of robustness to disorder at bent topological interfaces provides a rapid, flexible scheme for engineering high-performance topological photonic devices across multiple frequency regimes from microwave to THz, to visible.

When do languages use the same word for different meanings? The Goldilocks Principle in colexification

Lexical ambiguity is pervasive in language, and often systematic. For instance, the Spanish word "dedo" refers to both a toe and a finger, and this TOE-FINGER ambiguity is found in over 100 languages. Previous work shows that systematic ambiguities involve related meanings. This is attributed to cognitive pressure towards simplicity in language, as it makes lexicons easier to learn and use. The present study examines the interplay between this pressure and the competing pressure for languages to support accurate information transfer. We hypothesize that ambiguity follows a Goldilocks principle that balances the two pressures: meanings are more likely to attach to the same word when they are related to an optimal degree ---neither too much, nor too little. We find support for this principle in data from over 1200 languages and 1400 meanings. Our results thus suggest that universal forces shape the lexicons of natural languages.