scholarly journals On the benefits of representation regularization in invariance based domain generalization

2022 ◽  
Author(s):  
Changjian Shui ◽  
Boyu Wang ◽  
Christian Gagné
Keyword(s):  
Machine Learning ◽  
Theoretical Analysis ◽  
Regularization Method ◽  
Representation Learning ◽  
Crucial Aspect ◽  
Invariant Representation ◽  
Test Environment ◽  
Empirical Results ◽  
Empirical Performance

AbstractA crucial aspect of reliable machine learning is to design a deployable system for generalizing new related but unobserved environments. Domain generalization aims to alleviate such a prediction gap between the observed and unseen environments. Previous approaches commonly incorporated learning the invariant representation for achieving good empirical performance. In this paper, we reveal that merely learning the invariant representation is vulnerable to the related unseen environment. To this end, we derive a novel theoretical analysis to control the unseen test environment error in the representation learning, which highlights the importance of controlling the smoothness of representation. In practice, our analysis further inspires an efficient regularization method to improve the robustness in domain generalization. The proposed regularization is orthogonal to and can be straightforwardly adopted in existing domain generalization algorithms that ensure invariant representation learning. Empirical results show that our algorithm outperforms the base versions in various datasets and invariance criteria.

Digging for the truth: the case for active annotation in evaluating the credibility of online medical information (Preprint)

2020 ◽  
Author(s):  
Mikołaj Morzy ◽  
Bartłomiej Balcerzak ◽  
Adam Wierzbicki ◽  
Adam Wierzbicki
Keyword(s):  
Machine Learning ◽  
Medical Information ◽  
Representation Learning ◽  
Training Dataset ◽  
Highly Qualified ◽  
Human In The Loop ◽  
Annotation Process ◽  
Comprehensive Framework ◽  
Online Sources ◽  
The Web

BACKGROUND With the rapidly accelerating spread of dissemination of false medical information on the Web, the task of establishing the credibility of online sources of medical information becomes a pressing necessity. The sheer number of websites offering questionable medical information presented as reliable and actionable suggestions with possibly harmful effects poses an additional requirement for potential solutions, as they have to scale to the size of the problem. Machine learning is one such solution which, when properly deployed, can be an effective tool in fighting medical disinformation on the Web. OBJECTIVE We present a comprehensive framework for designing and curating of machine learning training datasets for online medical information credibility assessment. We show how the annotation process should be constructed and what pitfalls should be avoided. Our main objective is to provide researchers from medical and computer science communities with guidelines on how to construct datasets for machine learning models for various areas of medical information wars. METHODS The key component of our approach is the active annotation process. We begin by outlining the annotation protocol for the curation of high-quality training dataset, which then can be augmented and rapidly extended by employing the human-in-the-loop paradigm to machine learning training. To circumvent the cold start problem of insufficient gold standard annotations, we propose a pre-processing pipeline consisting of representation learning, clustering, and re-ranking of sentences for the acceleration of the training process and the optimization of human resources involved in the annotation. RESULTS We collect over 10 000 annotations of sentences related to selected subjects (psychiatry, cholesterol, autism, antibiotics, vaccines, steroids, birth methods, food allergy testing) for less than $7 000 employing 9 highly qualified annotators (certified medical professionals) and we release this dataset to the general public. We develop an active annotation framework for more efficient annotation of non-credible medical statements. The results of the qualitative analysis support our claims of the efficacy of the presented method. CONCLUSIONS A set of very diverse incentives is driving the widespread dissemination of medical disinformation on the Web. An effective strategy of countering this spread is to use machine learning for automatically establishing the credibility of online medical information. This, however, requires a thoughtful design of the training pipeline. In this paper we present a comprehensive framework of active annotation. In addition, we publish a large curated dataset of medical statements labelled as credible, non-credible, or neutral.

Structural, QSAR, machine learning and molecular docking studies of 5-thiophen-2-yl pyrazole derivatives as potent and selective cannabinoid-1 receptor antagonists

2021 ◽  
Author(s):  
Riad Hanachi ◽  
Ridha Ben Said ◽  
Hamza Allal ◽  
Seyfeddine Rahali ◽  
Mohammed A. M. Alkhalifah ◽  
...  
Keyword(s):  
Machine Learning ◽  
Biological Activity ◽  
Molecular Docking ◽  
Theoretical Analysis ◽  
Structural Study ◽  
Docking Studies ◽  
Pyrazole Derivatives ◽  
Molecular Docking Studies ◽  
Cannabinoid 1 Receptor ◽  
Chemical Descriptors

We performed a structural study followed by a theoretical analysis of the chemical descriptors and the biological activity of a series of 5-thiophen-2-yl pyrazole derivatives as potent and selective Cannabinoid-1...

scholarly journals An Exploratory Study of COVID-19 Information on Twitter in the Greater Region

2021 ◽  
Vol 5 (1) ◽  
pp. 5
Author(s):  
Ninghan Chen ◽  
Zhiqiang Zhong ◽  
Jun Pang
Keyword(s):  
Machine Learning ◽  
Social Networks ◽  
Online Social Networks ◽  
Exploratory Study ◽  
Representation Learning ◽  
Data Driven ◽  
Learning Methods ◽  
Twitter Users ◽  
Over Time

The outbreak of the COVID-19 led to a burst of information in major online social networks (OSNs). Facing this constantly changing situation, OSNs have become an essential platform for people expressing opinions and seeking up-to-the-minute information. Thus, discussions on OSNs may become a reflection of reality. This paper aims to figure out how Twitter users in the Greater Region (GR) and related countries react differently over time through conducting a data-driven exploratory study of COVID-19 information using machine learning and representation learning methods. We find that tweet volume and COVID-19 cases in GR and related countries are correlated, but this correlation only exists in a particular period of the pandemic. Moreover, we plot the changing of topics in each country and region from 22 January 2020 to 5 June 2020, figuring out the main differences between GR and related countries.

scholarly journals Machine learning outperforms thermodynamics in measuring how well a many-body system learns a drive

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Weishun Zhong ◽  
Jacob M. Gold ◽  
Sarah Marzen ◽  
Jeremy L. England ◽  
Nicole Yunger Halpern
Keyword(s):  
Machine Learning ◽  
Novelty Detection ◽  
Representation Learning ◽  
Many Body ◽  
Discrimination Ability ◽  
Machine Learning Model ◽  
Systems Learning ◽  
Statistical Mechanical ◽  
Far From Equilibrium ◽  
Many Body Systems

AbstractDiverse many-body systems, from soap bubbles to suspensions to polymers, learn and remember patterns in the drives that push them far from equilibrium. This learning may be leveraged for computation, memory, and engineering. Until now, many-body learning has been detected with thermodynamic properties, such as work absorption and strain. We progress beyond these macroscopic properties first defined for equilibrium contexts: We quantify statistical mechanical learning using representation learning, a machine-learning model in which information squeezes through a bottleneck. By calculating properties of the bottleneck, we measure four facets of many-body systems’ learning: classification ability, memory capacity, discrimination ability, and novelty detection. Numerical simulations of a classical spin glass illustrate our technique. This toolkit exposes self-organization that eludes detection by thermodynamic measures: Our toolkit more reliably and more precisely detects and quantifies learning by matter while providing a unifying framework for many-body learning.

scholarly journals Replay in minds and machines

2021 ◽  
Author(s):  
Lennart Wittkuhn ◽  
Samson Chien ◽  
Sam Hall-McMaster ◽  
Nicolas W. Schuck
Keyword(s):  
Machine Learning ◽  
Brain Activity ◽  
Activity Patterns ◽  
Representation Learning ◽  
Parallel Machine ◽  
Artificial Agents ◽  
Active Behavior ◽  
Performance Improvements ◽  
Experience Replay ◽  
Data Efficiency

Experience-related brain activity patterns have been found to reactivate during sleep, wakeful rest, and brief pauses from active behavior. In parallel, machine learning research has found that experience replay can lead to substantial performance improvements in artificial agents. Together, these lines of research have significantly expanded our understanding of the potential computational benefits replay may provide to biological and artificial agents alike. We provide an overview of findings in replay research from neuroscience and machine learning and summarize the computational benefits an agent can gain from replay that cannot be achieved through direct interactions with the world itself. These benefits include faster learning and data efficiency, less forgetting, prioritizing important experiences, as well as improved planning and generalization. In addition to the benefits of replay for improving an agent's decision-making policy, we highlight the less-well studied aspect of replay in representation learning, wherein replay could provide a mechanism to learn the structure and relevant aspects of the environment. Thus, replay might help the agent to build task-appropriate state representations.

scholarly journals Every Local Minimum Value Is the Global Minimum Value of Induced Model in Nonconvex Machine Learning

2019 ◽  
Vol 31 (12) ◽  
pp. 2293-2323 ◽  
Author(s):  
Kenji Kawaguchi ◽  
Jiaoyang Huang ◽  
Leslie Pack Kaelbling
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Local Minimum ◽  
Deep Neural Networks ◽  
Representation Learning ◽  
Minimum Value ◽  
Special Cases ◽  
Optimal Value ◽  
Special Case ◽  
Theoretical Results

For nonconvex optimization in machine learning, this article proves that every local minimum achieves the globally optimal value of the perturbable gradient basis model at any differentiable point. As a result, nonconvex machine learning is theoretically as supported as convex machine learning with a handcrafted basis in terms of the loss at differentiable local minima, except in the case when a preference is given to the handcrafted basis over the perturbable gradient basis. The proofs of these results are derived under mild assumptions. Accordingly, the proven results are directly applicable to many machine learning models, including practical deep neural networks, without any modification of practical methods. Furthermore, as special cases of our general results, this article improves or complements several state-of-the-art theoretical results on deep neural networks, deep residual networks, and overparameterized deep neural networks with a unified proof technique and novel geometric insights. A special case of our results also contributes to the theoretical foundation of representation learning.

Employee Contributions to Local Government Retirement Systems: A Theoretical Analysis of Effects

1980 ◽  
Vol 8 (4) ◽  
pp. 443-456
Author(s):  
Gene E. Mumy
Keyword(s):  
Theoretical Model ◽  
Local Government ◽  
Theoretical Analysis ◽  
Employee Compensation ◽  
Empirical Results ◽  
Retirement System ◽  
Retirement Systems ◽  
Theoretical Results

The purpose of this article is to construct a theoretical model capable of assessing the extent to which required employee contributions to a local government retirement system necessitate increased levels of employee compensation. It is found that increased compensation is required if the before-tax yield on retirement system assets is less than the before-tax yield on alternative assets that would be held by employees. It is also found that such increased levels of compensation generally take the form of increased pensions rather than increased wages. These theoretical results are compared with some recent empirical results obtained by Ehrenberg.

scholarly journals Customized Dictionary Learning for Subdatasets with Fine Granularity

2016 ◽  
Vol 2016 ◽  
pp. 1-13
Author(s):  
Lei Ye ◽  
Can Wang ◽  
Xin Xu ◽  
Hui Qian
Keyword(s):  
Machine Learning ◽  
Computer Vision ◽  
Theoretical Analysis ◽  
Improve Performance ◽  
Real World Data ◽  
Sparse Models ◽  
Learned Dictionary ◽  
Wide Range ◽  
Sparsity Constraints ◽  
The Difference

Sparse models have a wide range of applications in machine learning and computer vision. Using a learned dictionary instead of an “off-the-shelf” one can dramatically improve performance on a particular dataset. However, learning a new one for each subdataset (subject) with fine granularity may be unwarranted or impractical, due to restricted availability subdataset samples and tremendous numbers of subjects. To remedy this, we consider the dictionary customization problem, that is, specializing an existing global dictionary corresponding to the total dataset, with the aid of auxiliary samples obtained from the target subdataset. Inspired by observation and then deduced from theoretical analysis, a regularizer is employed penalizing the difference between the global and the customized dictionary. By minimizing the sum of reconstruction errors of the above regularizer under sparsity constraints, we exploit the characteristics of the target subdataset contained in the auxiliary samples while maintaining the basic sketches stored in the global dictionary. An efficient algorithm is presented and validated with experiments on real-world data.

scholarly journals Neural generative models and representation learning for information retrieval

2019 ◽  
Vol 53 (2) ◽  
pp. 97-97
Author(s):  
Qingyao Ai
Keyword(s):  
Information Retrieval ◽  
Theoretical Analysis ◽  
Language Processing ◽  
Ad Hoc ◽  
Representation Learning ◽  
Generative Models ◽  
Neural Models ◽  
Retrieval Models ◽  
Types Of Information ◽  
Text Images

Information Retrieval (IR) concerns about the structure, analysis, organization, storage, and retrieval of information. Among different retrieval models proposed in the past decades, generative retrieval models, especially those under the statistical probabilistic framework, are one of the most popular techniques that have been widely applied to Information Retrieval problems. While they are famous for their well-grounded theory and good empirical performance in text retrieval, their applications in IR are often limited by their complexity and low extendability in the modeling of high-dimensional information. Recently, advances in deep learning techniques provide new opportunities for representation learning and generative models for information retrieval. In contrast to statistical models, neural models have much more flexibility because they model information and data correlation in latent spaces without explicitly relying on any prior knowledge. Previous studies on pattern recognition and natural language processing have shown that semantically meaningful representations of text, images, and many types of information can be acquired with neural models through supervised or unsupervised training. Nonetheless, the effectiveness of neural models for information retrieval is mostly unexplored. In this thesis, we study how to develop new generative models and representation learning frameworks with neural models for information retrieval. Specifically, our contributions include three main components: (1) Theoretical Analysis : We present the first theoretical analysis and adaptation of existing neural embedding models for ad-hoc retrieval tasks; (2) Design Practice : Based on our experience and knowledge, we show how to design an embedding-based neural generative model for practical information retrieval tasks such as personalized product search; And (3) Generic Framework : We further generalize our proposed neural generative framework for complicated heterogeneous information retrieval scenarios that concern text, images, knowledge entities, and their relationships. Empirical results show that the proposed neural generative framework can effectively learn information representations and construct retrieval models that outperform the state-of-the-art systems in a variety of IR tasks.

scholarly journals Heterogeneous Network Representation Learning

2020 ◽  
Author(s):  
Yuxiao Dong ◽  
Ziniu Hu ◽  
Kuansan Wang ◽  
Yizhou Sun ◽  
Jie Tang
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Heterogeneous Network ◽  
Representation Learning ◽  
Machine Learning Algorithms ◽  
Future Directions ◽  
Relational Properties ◽  
Open Research ◽  
Different Types ◽  
Graph Neural Networks

Representation learning has offered a revolutionary learning paradigm for various AI domains. In this survey, we examine and review the problem of representation learning with the focus on heterogeneous networks, which consists of different types of vertices and relations. The goal of this problem is to automatically project objects, most commonly, vertices, in an input heterogeneous network into a latent embedding space such that both the structural and relational properties of the network can be encoded and preserved. The embeddings (representations) can be then used as the features to machine learning algorithms for addressing corresponding network tasks. To learn expressive embeddings, current research developments can fall into two major categories: shallow embedding learning and graph neural networks. After a thorough review of the existing literature, we identify several critical challenges that remain unaddressed and discuss future directions. Finally, we build the Heterogeneous Graph Benchmark to facilitate open research for this rapidly-developing topic.

Sign in / Sign up

Export Citation Format

Share Document