Inductive Contextual Relation Learning for Personalization

2021 ◽  
Vol 39 (3) ◽  
pp. 1-22
Author(s):  
Chuxu Zhang ◽  
Huaxiu Yao ◽  
Lu Yu ◽  
Chao Huang ◽  
Dongjin Song ◽  
...  
Keyword(s):  
Gradient Descent ◽  
Inductive Learning ◽  
Model Parameters ◽  
Learning Approaches ◽  
Web Personalization ◽  
Optimization Scheme ◽  
Smoothness Constraint ◽  
Contextual Relation ◽  
Relation Learning ◽  
New Item

Web personalization, e.g., recommendation or relevance search, tailoring a service/product to accommodate specific online users, is becoming increasingly important. Inductive personalization aims to infer the relations between existing entities and unseen new ones, e.g., searching relevant authors for new papers or recommending new items to users. This problem, however, is challenging since most of recent studies focus on transductive problem for existing entities. In addition, despite some inductive learning approaches have been introduced recently, their performance is sub-optimal due to relatively simple and inflexible architectures for aggregating entity’s content. To this end, we propose the inductive contextual personalization (ICP) framework through contextual relation learning. Specifically, we first formulate the pairwise relations between entities with a ranking optimization scheme that employs neural aggregator to fuse entity’s heterogeneous contents. Next, we introduce a node embedding term to capture entity’s contextual relations, as a smoothness constraint over the prior ranking objective. Finally, the gradient descent procedure with adaptive negative sampling is employed to learn the model parameters. The learned model is capable of inferring the relations between existing entities and inductive ones. Thorough experiments demonstrate that ICP outperforms numerous baseline methods for two different applications, i.e., relevant author search and new item recommendation.

scholarly journals Dynamic Detection of Delayed Cerebral Ischemia Using Machine Learning

2020 ◽  
Author(s):  
Murad Megjhani ◽  
Kalijah Terilli ◽  
Ayham Alkhachroum ◽  
David J. Roh ◽  
Sachin Agarwal ◽  
...  
Keyword(s):  
Machine Learning ◽  
Cerebral Ischemia ◽  
Characteristic Curve ◽  
Delayed Cerebral Ischemia ◽  
Risk Scores ◽  
Support Vector ◽  
Model Parameters ◽  
Learning Approaches ◽  
Physiologic Data ◽  
Over Time

AbstractObjectiveTo develop a machine learning based tool, using routine vital signs, to assess delayed cerebral ischemia (DCI) risk over time.MethodsIn this retrospective analysis, physiologic data for 540 consecutive acute subarachnoid hemorrhage patients were collected and annotated as part of a prospective observational cohort study between May 2006 and December 2014. Patients were excluded if (i) no physiologic data was available, (ii) they expired prior to the DCI onset window (< post bleed day 3) or (iii) early angiographic vasospasm was detected on admitting angiogram. DCI was prospectively labeled by consensus of treating physicians. Occurrence of DCI was classified using various machine learning approaches including logistic regression, random forest, support vector machine (linear and kernel), and an ensemble classifier, trained on vitals and subject characteristic features. Hourly risk scores were generated as the posterior probability at time t. We performed five-fold nested cross validation to tune the model parameters and to report the accuracy. All classifiers were evaluated for good discrimination using the area under the receiver operating characteristic curve (AU-ROC) and confusion matrices.ResultsOf 310 patients included in our final analysis, 101 (32.6%) patients developed DCI. We achieved maximal classification of 0.81 [0.75-0.82] AU-ROC. We also predicted 74.7 % of all DCI events 12 hours before typical clinical detection with a ratio of 3 true alerts for every 2 false alerts.ConclusionA data-driven machine learning based detection tool offered hourly assessments of DCI risk and incorporated new physiologic information over time.

scholarly journals Self-Supervised Pre-Training of Transformers for Satellite Image Time Series Classification

2020 ◽  
Author(s):  
Yuan Yuan ◽  
Lei Lin
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Large Scale ◽  
Temporal Structure ◽  
Satellite Image ◽  
Fine Tuning ◽  
Small Scale ◽  
Model Parameters ◽  
Learning Approaches ◽  
Wide Range

Satellite image time series (SITS) classification is a major research topic in remote sensing and is relevant for a wide range of applications. Deep learning approaches have been commonly employed for SITS classification and have provided state-of-the-art performance. However, deep learning methods suffer from overfitting when labeled data is scarce. To address this problem, we propose a novel self-supervised pre-training scheme to initialize a Transformer-based network by utilizing large-scale unlabeled data. In detail, the model is asked to predict randomly contaminated observations given an entire time series of a pixel. The main idea of our proposal is to leverage the inherent temporal structure of satellite time series to learn general-purpose spectral-temporal representations related to land cover semantics. Once pre-training is completed, the pre-trained network can be further adapted to various SITS classification tasks by fine-tuning all the model parameters on small-scale task-related labeled data. In this way, the general knowledge and representations about SITS can be transferred to a label-scarce task, thereby improving the generalization performance of the model as well as reducing the risk of overfitting. Comprehensive experiments have been carried out on three benchmark datasets over large study areas. Experimental results demonstrate the effectiveness of the proposed method, leading to a classification accuracy increment up to 1.91% to 6.69%. <div><b>This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.</b></div>

Efficient inference for agent-based models of real-world phenomena

2021 ◽  
Author(s):  
Andreas Christ Sølvsten Jørgensen ◽  
Atiyo Ghosh ◽  
Marc Sturrock ◽  
Vahid Shahrezaei
Keyword(s):  
Machine Learning ◽  
Case Studies ◽  
Parameter Space ◽  
Real World ◽  
Autonomous Agents ◽  
Stochastic Simulations ◽  
Model Parameters ◽  
Learning Approaches ◽  
Real World Applications ◽  
Real World Problems

AbstractThe modelling of many real-world problems relies on computationally heavy simulations. Since statistical inference rests on repeated simulations to sample the parameter space, the high computational expense of these simulations can become a stumbling block. In this paper, we compare two ways to mitigate this issue based on machine learning methods. One approach is to construct lightweight surrogate models to substitute the simulations used in inference. Alternatively, one might altogether circumnavigate the need for Bayesian sampling schemes and directly estimate the posterior distribution. We focus on stochastic simulations that track autonomous agents and present two case studies of real-world applications: tumour growths and the spread of infectious diseases. We demonstrate that good accuracy in inference can be achieved with a relatively small number of simulations, making our machine learning approaches orders of magnitude faster than classical simulation-based methods that rely on sampling the parameter space. However, we find that while some methods generally produce more robust results than others, no algorithm offers a one-size-fits-all solution when attempting to infer model parameters from observations. Instead, one must choose the inference technique with the specific real-world application in mind. The stochastic nature of the considered real-world phenomena poses an additional challenge that can become insurmountable for some approaches. Overall, we find machine learning approaches that create direct inference machines to be promising for real-world applications. We present our findings as general guidelines for modelling practitioners.Author summaryComputer simulations play a vital role in modern science as they are commonly used to compare theory with observations. One can thus infer the properties of a observed system by comparing the data to the predicted behaviour in different scenarios. Each of these scenarios corresponds to a simulation with slightly different settings. However, since real-world problems are highly complex, the simulations often require extensive computational resources, making direct comparisons with data challenging, if not insurmountable. It is, therefore, necessary to resort to inference methods that mitigate this issue, but it is not clear-cut what path to choose for any specific research problem. In this paper, we provide general guidelines for how to make this choice. We do so by studying examples from oncology and epidemiology and by taking advantage of developments in machine learning. More specifically, we focus on simulations that track the behaviour of autonomous agents, such as single cells or individuals. We show that the best way forward is problem-dependent and highlight the methods that yield the most robust results across the different case studies. We demonstrate that these methods are highly promising and produce reliable results in a small fraction of the time required by classic approaches that rely on comparisons between data and individual simulations. Rather than relying on a single inference technique, we recommend employing several methods and selecting the most reliable based on predetermined criteria.

scholarly journals Application of Machine Learning to Bending Processes and Material Identification

Metals ◽  
2021 ◽  
Vol 11 (9) ◽  
pp. 1418
Author(s):  
Daniel J. Cruz ◽  
Manuel R. Barbosa ◽  
Abel D. Santos ◽  
Sara S. Miranda ◽  
Rui L. Amaral
Keyword(s):  
Machine Learning ◽  
Sheet Metal ◽  
Bending Test ◽  
Model Parameters ◽  
Learning Approaches ◽  
Additional Information ◽  
The Many ◽  
And Control ◽  
Force Displacement ◽  
Springback Angle

The increasing availability of data, which becomes a continually increasing trend in multiple fields of application, has given machine learning approaches a renewed interest in recent years. Accordingly, manufacturing processes and sheet metal forming follow such directions, having in mind the efficiency and control of the many parameters involved, in processing and material characterization. In this article, two applications are considered to explore the capability of machine learning modeling through shallow artificial neural networks (ANN). One consists of developing an ANN to identify the constitutive model parameters of a material using the force–displacement curves obtained with a standard bending test. The second one concentrates on the springback problem in sheet metal press-brake air bending, with the objective of predicting the punch displacement required to attain a desired bending angle, including additional information of the springback angle. The required data for designing the ANN solutions are collected from numerical simulation using finite element methodology (FEM), which in turn was validated by experiments.

scholarly journals Self-Supervised Pre-Training of Transformers for Satellite Image Time Series Classification

2020 ◽  
Author(s):  
Yuan Yuan ◽  
Lei Lin
Keyword(s):  
Time Series ◽  
Deep Learning ◽  
Large Scale ◽  
Temporal Structure ◽  
Satellite Image ◽  
Fine Tuning ◽  
Small Scale ◽  
Model Parameters ◽  
Learning Approaches ◽  
Wide Range

<div>Satellite image time series (SITS) classification is a major research topic in remote sensing and is relevant for a wide range of applications. Deep learning approaches have been commonly employed for SITS classification and have provided state-of-the-art performance. However, deep learning methods suffer from overfitting when labeled data is scarce. To address this problem, we propose a novel self-supervised pre-training scheme to initialize a Transformer-based network by utilizing large-scale unlabeled data. In detail, the model is asked to predict randomly contaminated observations given an entire time series of a pixel. The main idea of our proposal is to leverage the inherent temporal structure of satellite time series to learn general-purpose spectral-temporal representations related to land cover semantics. Once pre-training is completed, the pre-trained network can be further adapted to various SITS classification tasks by fine-tuning all the model parameters on small-scale task-related labeled data. In this way, the general knowledge and representations about SITS can be transferred to a label-scarce task, thereby improving the generalization performance of the model as well as reducing the risk of overfitting. Comprehensive experiments have been carried out on three benchmark datasets over large study areas. Experimental results demonstrate the effectiveness of the proposed method, leading to a classification accuracy increment up to 2.38% to 5.27%. The code and the pre-trained model will be available at https://github.com/linlei1214/SITS-BERT upon publication.</div><div><b>This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.</b></div>

scholarly journals Learning SKOS Relations for Terminological Ontologies from Text

2011 ◽  
pp. 129-152 ◽  
Author(s):  
Wei Wang ◽  
Payam M. Barnaghi ◽  
Andrzej Bargiela
Keyword(s):  
Learning Algorithm ◽  
Topic Models ◽  
Quantitative Measure ◽  
Model Parameters ◽  
Domain Experts ◽  
Concept Extraction ◽  
Probabilistic Topic Models ◽  
Novel Approach ◽  
Relation Learning ◽  
Concept Hierarchies

The problem of learning concept hierarchies and terminological ontologies can be divided into two sub-tasks: concept extraction and relation learning. The authors of this chapter describe a novel approach to learn relations automatically from unstructured text corpus based on probabilistic topic models. The authors provide definition (Information Theory Principle for Concept Relationship) and quantitative measure for establishing “broader” (or “narrower”) and “related” relations between concepts. They present a relation learning algorithm to automatically interconnect concepts into concept hierarchies and terminological ontologies with the probabilistic topic models learned. In this experiment, around 7,000 ontology statements expressed in terms of “broader” and “related” relations are generated using different combination of model parameters. The ontology statements are evaluated by domain experts and the results show that the highest precision of the learned ontologies is around 86.6% and structures of learned ontologies remain stable when values of the parameters are changed in the ontology learning algorithm.

Iterative gradient descent for outlier detection

2021 ◽  
pp. 2150004
Author(s):  
Zhuang Qi ◽  
Dazhi Jiang ◽  
Xiaming Chen
Keyword(s):  
Outlier Detection ◽  
Gradient Descent ◽  
Robust Regression ◽  
Mean Shift ◽  
Information Criterion ◽  
Descent Method ◽  
Parameter Selection ◽  
Model Parameters ◽  
Gradient Descent Method ◽  
Key Steps

In linear regression, outliers have a serious effect on the estimation of regression model parameters and the prediction of final results, so outlier detection is one of the key steps in data analysis. In this paper, we use a mean shift model and then we apply the penalty function to penalize the mean shift parameters, which is conducive to get a sparse parameter vector. We choose Sorted L1 regularization (SLOPE), which provides a convex loss function, and shows good statistical properties in parameter selection. We apply an iterative process which using gradient descent method and parameter selection at each step. Our algorithm has higher computational efficiency since the calculation of inverse matrix is avoided. Finally, we use Cross-Validation rules (CV) and Bayesian Information Criterion (BIC) criteria to fine tune the parameters, which helps our program identify outliers and obtain more robust regression coefficients. Compared with other methods, the experimental results show that our program has a fantastic performance in all aspects of outlier detection.

scholarly journals Inductive Learning Approaches for Improving Pilot Awareness of Aircraft Faults

2005 ◽  
Author(s):  
Lilly Spirkovska ◽  
David Iverson ◽  
Scott Poll ◽  
Anna Pryor
Keyword(s):  
Inductive Learning ◽  
Learning Approaches

scholarly journals Distributed Stochastic Gradient Descent with Event-Triggered Communication

2020 ◽  
Vol 34 (05) ◽  
pp. 7169-7178 ◽  
Author(s):  
Jemin George ◽  
Prudhvi Gurram
Keyword(s):  
Gradient Descent ◽  
Optimization Problems ◽  
Sufficient Conditions ◽  
Stochastic Gradient ◽  
Training Data ◽  
Stochastic Gradient Descent ◽  
Model Parameters ◽  
Trained Neural Network ◽  
The Individual ◽  
Event Triggered

We develop a Distributed Event-Triggered Stochastic GRAdient Descent (DETSGRAD) algorithm for solving non-convex optimization problems typically encountered in distributed deep learning. We propose a novel communication triggering mechanism that would allow the networked agents to update their model parameters aperiodically and provide sufficient conditions on the algorithm step-sizes that guarantee the asymptotic mean-square convergence. The algorithm is applied to a distributed supervised-learning problem, in which a set of networked agents collaboratively train their individual neural networks to perform image classification, while aperiodically sharing the model parameters with their one-hop neighbors. Results indicate that all agents report similar performance that is also comparable to the performance of a centrally trained neural network, while the event-triggered communication provides significant reduction in inter-agent communication. Results also show that the proposed algorithm allows the individual agents to classify the images even though the training data corresponding to all the classes are not locally available to each agent.

Sign in / Sign up

Export Citation Format

Share Document