Knowledge Preserving and Distribution Alignment for Heterogeneous Domain Adaptation

Domain adaptation aims at improving the performance of learning tasks in a target domain by leveraging the knowledge extracted from a source domain. To this end, one can perform knowledge transfer between these two domains. However, this problem becomes extremely challenging when the data of these two domains are characterized by different types of features, i.e., the feature spaces of the source and target domains are different, which is referred to as heterogeneous domain adaptation (HDA). To solve this problem, we propose a novel model called Knowledge Preserving and Distribution Alignment (KPDA), which learns an augmented target space by jointly minimizing information loss and maximizing domain distribution alignment. Specifically, we seek to discover a latent space, where the knowledge is preserved by exploiting the Laplacian graph terms and reconstruction regularizations. Moreover, we adopt the Maximum Mean Discrepancy to align the distributions of the source and target domains in the latent space. Mathematically, KPDA is formulated as a minimization problem with orthogonal constraints, which involves two projection variables. Then, we develop an algorithm based on the Gauss–Seidel iteration scheme and split the problem into two subproblems, which are solved by searching algorithms based on the Barzilai–Borwein (BB) stepsize. Promising results demonstrate the effectiveness of the proposed method.

Optical neuromorphic processing with soliton crystal Kerr microcombs: Scaling the network in size and speed

Convolutional neural networks (CNNs), inspired by biological visual cortex systems, are a powerful category of artificial neural networks that can extract the hierarchical features of raw data to greatly reduce the network parametric complexity and enhance the predicting accuracy. They are of significant interest for machine learning tasks such as computer vision, speech recognition, playing board games and medical diagnosis [1-7]. Optical neural networks offer the promise of dramatically accelerating computing speed to overcome the inherent bandwidth bottleneck of electronics. Here, we demonstrate a universal optical vector convolutional accelerator operating beyond 10 Tera-OPS (TOPS - operations per second), generating convolutions of images of 250,000 pixels with 8-bit resolution for 10 kernels simultaneously — enough for facial image recognition. We then use the same hardware to sequentially form a deep optical CNN with ten output neurons, achieving successful recognition of full 10 digits with 900 pixel handwritten digit images with 88% accuracy. Our results are based on simultaneously interleaving temporal, wavelength and spatial dimensions enabled by an integrated microcomb source. We show that this approach is scalable and trainable to much more complex networks for demanding applications such as unmanned vehicle and real-time video recognition.Keywords: Optical neural networks, neuromorphic processor, microcomb, convolutional accelerator

A Novel Data Analytics Oriented Approach for Image Representation Learning in Manufacturing Systems

In the era of digital manufacturing, huge amount of image data generated by manufacturing systems cannot be instantly handled to obtain valuable information due to the limitations (e.g., time) of traditional techniques of image processing. In this paper, we propose a novel self-supervised self-attention learning framework—TriLFrame for image representation learning. The TriLFrame is based on the hybrid architecture of Convolutional Network and Transformer. Experiments show that TriLFrame outperforms state-of-the-art self-supervised methods on the ImageNet dataset and achieves competitive performances when transferring learned features on ImageNet to other classification tasks. Moreover, TriLFrame verifies the proposed hybrid architecture, which combines the powerful local convolutional operation and the long-range nonlocal self-attention operation and works effectively in image representation learning tasks.

Class Highlight Generative Adversarial Networks for Strip Steel Defect Classification

In order to solve the problem of dataset expansion in deep learning tasks such as image classification, this paper proposed an image generation model called Class Highlight Generative Adversarial Networks (CH-GANs). In order to highlight image categories, accelerate the convergence speed of the model and generate true-to-life images with clear categories, first, the image category labels were deconvoluted and integrated into the generator through [Formula: see text] convolution. Second, a novel discriminator that cannot only judge the authenticity of the image but also the image category was designed. Finally, in order to quickly and accurately classify strip steel defects, the lightweight image classification network GhostNet was appropriately improved by modifying the number of network layers and the number of network channels, adding SE modules, etc., and was trained on the dataset expanded by CH-GAN. In the comparative experiments, the average FID of CH-GAN is 7.59; the accuracy of the improved GhostNet is 95.67% with 0.19[Formula: see text]M parameters. The experimental results prove the effectiveness and superiority of the methods proposed in this paper in the generation and classification of strip steel defect images.

Confirmation Bias in the Course of Instructed Reinforcement Learning in Schizophrenia-Spectrum Disorders

A large body of research attributes learning deficits in schizophrenia (SZ) to the systems involved in value representation (prefrontal cortex, PFC) and reinforcement learning (basal ganglia, BG) as well as to the compromised connectivity of these regions. In this study, we employed learning tasks hypothesized to probe the function and interaction of the PFC and BG in patients with SZ-spectrum disorders in comparison to healthy control (HC) subjects. In the Instructed Probabilistic Selection task (IPST), participants received false instruction about one of the stimuli used in the course of probabilistic learning which creates confirmation bias, whereby the instructed stimulus is overvalued in comparison to its real experienced value. The IPST was administered to 102 patients with SZ and 120 HC subjects. We have shown that SZ patients and HC subjects were equally influenced by false instruction in reinforcement learning (RL) probabilistic task (IPST) (p-value = 0.441); however, HC subjects had significantly higher learning rates associated with the process of overcoming cognitive bias in comparison to SZ patients (p-value = 0.018). The behavioral results of our study could be hypothesized to provide further evidence for impairments in the SZ-BG circuitry; however, this should be verified by neurofunctional imaging studies.

Achieving 90% In Data-Centric Industry Deep Learning Task

In industry deep learning application, our manually labeled data has a certain number of noisy data. To solve this problem and achieve more than 90 score in dev dataset, we present a simple method to find the noisy data and re-label the noisy data by human, given the model predictions as references in human labeling. In this paper, we illustrate our idea for a broad set of deep learning tasks, includes classification, sequence tagging, object detection, sequence generation, click-through rate prediction. The experimental results and human evaluation results verify our idea.

An Introduction to the Federated Learning Standard

With the growing concern on data privacy and security, it is undesirable to collect data from all users to perform machine learning tasks. Federated learning, a decentralized learning framework, was proposed to construct a shared prediction model while keeping owners' data on their own devices. This paper presents an introduction to the emerging federated learning standard and discusses its various aspects, including i) an overview of federated learning, ii) types of federated learning, iii) major concerns and the performance evaluation criteria of federated learning, and iv) associated regulatory requirements. The purpose of this paper is to provide an understanding of the standard and facilitate its usage in model building across organizations while meeting privacy and security concerns.

Communication in the Context of COVID

Effective communication in the context of COVID-19 is the key element that underlies school success, consisting of a complex repertoire of intentional actions capable of producing positive changes or transformations in the field of cognition, affect, and behavior, on the personality as a whole. The communication aims to be convincing by students taking over an idea, behavior, or attitude based on arguments, realistic evidence, and to determine involvement, trust, openness, and dedication in achieving the goal and persuasive through the ability to change certain ideas or behaviors through logos, pathos, and ethos. The teacher has not only the role of information, but also to communicate with the children in his class about values, ideals, attitudes with a role in their formation and development. Feedback becomes effective when students show intrinsic involvement in learning tasks.

A Cognitive Load Perspective to Instructional Design for Online Learning

Learners struggle to keep up with the cognitive demands of online learning. Terms referring to the drain of learners' cognitive resources such as “Zoom fatigue” have been around for a while. The instructional design of online courses must consider cognitive factors more than ever. The cognitive load theory (CLT) has major underpinnings for designing online courses. The CLT seeks to optimize the learning process by considering the demands of the learning tasks (intrinsic cognitive load), design of the learning material (extrinsic cognitive load), and activation of learners' cognitive resources (germane cognitive load). Several principles have been proposed to manage each cognitive load type. This chapter will begin by outlining the CLT. Then, well-defined cognitive load effects will be introduced, along with evidence from the field. Next, new frontiers of the theory will be presented. Finally, implications of the cognitive load effects for online learning practices will be discussed.

An approach framework of transfer learning, adversarial training and hierarchical multi-task learning - a case study of disinformation detection with offensive text

With society going online and disinformation getting accepted as a phenomena that we have to live with, there is a growing need to automatically detect offensive text on modern social media platforms. But the lack of enough balanced labeled data, constantly evolving socio-linguistic patterns and ever-changing definition of offensive text make it a challenging task. This is a common pattern witnessed in all disinformation detection tasks such as detection of propaganda, rumour, fake news, hate etc. The work described in this paper improves upon the existing body of techniques by bringing in an approach framework that can surpass the existing benchmarks. Firstly, it addresses the imbalanced and insufficient nature of available labeled dataset. Secondly, learning using relates tasks through multi-task learning has been proved to be an effective approach in this domain but it has the unrealistic requirement of labeled data for all related tasks. The framework presented here suitably uses transfer learning in lieu of multi-task learning to address this issue. Thirdly, it builds a model explicitly addressing the hierarchical nature in the taxonomy of disinformation being detected as that delivers a stronger error feedback to the learning tasks. Finally, the model is made more robust by adversarial training. The work presented in this paper uses offensive text detection as a case study and shows convincing results for the chosen approach. The framework adopted can be easily replicated in other similar learning tasks facing a similar set of challenges.