scholarly journals Adaptive Scheduling for Machine Learning Tasks over Networks

2021 ◽  
Author(s):  
Konstantinos Gatsis
Keyword(s):  
Machine Learning ◽  
Adaptive Scheduling ◽  
Learning Tasks

scholarly journals TPOT-NN: augmenting tree-based automated machine learning with neural network estimators

2021 ◽  
Author(s):  
Joseph D. Romano ◽  
Trang T. Le ◽  
Weixuan Fu ◽  
Jason H. Moore
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Binary Classification ◽  
Inductive Learning ◽  
Future Directions ◽  
High Performing ◽  
Learning Tasks ◽  
Benchmark Datasets ◽  
Automated Machine Learning ◽  
Standard Tree

AbstractAutomated machine learning (AutoML) and artificial neural networks (ANNs) have revolutionized the field of artificial intelligence by yielding incredibly high-performing models to solve a myriad of inductive learning tasks. In spite of their successes, little guidance exists on when to use one versus the other. Furthermore, relatively few tools exist that allow the integration of both AutoML and ANNs in the same analysis to yield results combining both of their strengths. Here, we present TPOT-NN—a new extension to the tree-based AutoML software TPOT—and use it to explore the behavior of automated machine learning augmented with neural network estimators (AutoML+NN), particularly when compared to non-NN AutoML in the context of simple binary classification on a number of public benchmark datasets. Our observations suggest that TPOT-NN is an effective tool that achieves greater classification accuracy than standard tree-based AutoML on some datasets, with no loss in accuracy on others. We also provide preliminary guidelines for performing AutoML+NN analyses, and recommend possible future directions for AutoML+NN methods research, especially in the context of TPOT.

scholarly journals Representing Deep Neural Networks Latent Space Geometries with Graphs

Algorithms ◽  
2021 ◽  
Vol 14 (2) ◽  
pp. 39
Author(s):  
Carlos Lassance ◽  
Vincent Gripon ◽  
Antonio Ortega
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Objective Function ◽  
Learning Process ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
The Core ◽  
Learning Tasks ◽  
Latent Space

Deep Learning (DL) has attracted a lot of attention for its ability to reach state-of-the-art performance in many machine learning tasks. The core principle of DL methods consists of training composite architectures in an end-to-end fashion, where inputs are associated with outputs trained to optimize an objective function. Because of their compositional nature, DL architectures naturally exhibit several intermediate representations of the inputs, which belong to so-called latent spaces. When treated individually, these intermediate representations are most of the time unconstrained during the learning process, as it is unclear which properties should be favored. However, when processing a batch of inputs concurrently, the corresponding set of intermediate representations exhibit relations (what we call a geometry) on which desired properties can be sought. In this work, we show that it is possible to introduce constraints on these latent geometries to address various problems. In more detail, we propose to represent geometries by constructing similarity graphs from the intermediate representations obtained when processing a batch of inputs. By constraining these Latent Geometry Graphs (LGGs), we address the three following problems: (i) reproducing the behavior of a teacher architecture is achieved by mimicking its geometry, (ii) designing efficient embeddings for classification is achieved by targeting specific geometries, and (iii) robustness to deviations on inputs is achieved via enforcing smooth variation of geometry between consecutive latent spaces. Using standard vision benchmarks, we demonstrate the ability of the proposed geometry-based methods in solving the considered problems.

A content spectral-based text representation

2021 ◽  
pp. 1-12
Author(s):  
Melesio Crespo-Sanchez ◽  
Ivan Lopez-Arevalo ◽  
Edwin Aldana-Bobadilla ◽  
Alejandro Molina-Villegas
Keyword(s):  
Machine Learning ◽  
Text Analysis ◽  
Question Answering ◽  
Learning Algorithms ◽  
Machine Learning Algorithms ◽  
Text Representation ◽  
Feature Vectors ◽  
Learning Tasks ◽  
Semantic Component ◽  
Vector Representations

In the last few years, text analysis has grown as a keystone in several domains for solving many real-world problems, such as machine translation, spam detection, and question answering, to mention a few. Many of these tasks can be approached by means of machine learning algorithms. Most of these algorithms take as input a transformation of the text in the form of feature vectors containing an abstraction of the content. Most of recent vector representations focus on the semantic component of text, however, we consider that also taking into account the lexical and syntactic components the abstraction of content could be beneficial for learning tasks. In this work, we propose a content spectral-based text representation applicable to machine learning algorithms for text analysis. This representation integrates the spectra from the lexical, syntactic, and semantic components of text producing an abstract image, which can also be treated by both, text and image learning algorithms. These components came from feature vectors of text. For demonstrating the goodness of our proposal, this was tested on text classification and complexity reading score prediction tasks obtaining promising results.

scholarly journals A REVIEW OF FEATURE EXTRACTION METHODS ON MACHINE LEARNING

2021 ◽  
Vol 6 (22) ◽  
pp. 51-59
Author(s):  
Mustazzihim Suhaidi ◽  
Rabiah Abdul Kadir ◽  
Sabrina Tiun
Keyword(s):  
Machine Learning ◽  
Feature Extraction ◽  
Feature Selection ◽  
Input Data ◽  
Feature Vector ◽  
Learning Algorithm ◽  
Extraction Methods ◽  
Machine Learning Algorithm ◽  
Learning Tasks ◽  
Low Dimensional

Extracting features from input data is vital for successful classification and machine learning tasks. Classification is the process of declaring an object into one of the predefined categories. Many different feature selection and feature extraction methods exist, and they are being widely used. Feature extraction, obviously, is a transformation of large input data into a low dimensional feature vector, which is an input to classification or a machine learning algorithm. The task of feature extraction has major challenges, which will be discussed in this paper. The challenge is to learn and extract knowledge from text datasets to make correct decisions. The objective of this paper is to give an overview of methods used in feature extraction for various applications, with a dataset containing a collection of texts taken from social media.

scholarly journals A Simple Ensemble Learning Knowledge Distillation

2020 ◽  
Author(s):  
Himel Das Gupta ◽  
Kun Zhang ◽  
Victor S. Sheng
Keyword(s):  
Machine Learning ◽  
Ensemble Learning ◽  
Performance Enhancement ◽  
Portable Devices ◽  
Computational Power ◽  
Small Machine ◽  
Learning Tasks ◽  
Machine Learning Applications ◽  
Knowledge Distillation ◽  
Small Models

Deep neural network (DNN) has shown significant improvement in learning and generalizing different machine learning tasks over the years. But it comes with an expense of heavy computational power and memory requirements. We can see that machine learning applications are even running in portable devices like mobiles and embedded systems nowadays, which generally have limited resources regarding computational power and memory and thus can only run small machine learning models. However, smaller networks usually do not perform very well. In this paper, we have implemented a simple ensemble learning based knowledge distillation network to improve the accuracy of such small models. Our experimental results prove that the performance enhancement of smaller models can be achieved through distilling knowledge from a combination of small models rather than using a cumbersome model for the knowledge transfer. Besides, the ensemble knowledge distillation network is simpler, time-efficient, and easy to implement.

scholarly journals JAMPI: Efficient Matrix Multiplication in Spark Using Barrier Execution Mode

2020 ◽  
Author(s):  
Tamas Foldi ◽  
Chris von Csefalvay ◽  
Nicolas A. Perez
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Message Passing ◽  
Matrix Multiplication ◽  
Map Reduce ◽  
Distributed Training ◽  
Learning Tasks ◽  
Memory Footprint ◽  
Asynchronous Network ◽  
Execution Mode

The new barrier mode in Apache Spark allows embedding distributed deep learning training as a Spark stage to simplify the distributed training workflow. In Spark, a task in a stage doesn’t depend on any other tasks in the same stage, and hence it can be scheduled independently. However, several algorithms require more sophisticated inter-task communications, similar to the MPI paradigm. By combining distributed message passing (using asynchronous network IO), OpenJDK’s new auto-vectorization and Spark’s barrier execution mode, we can add non-map/reduce based algorithms, such as Cannon’s distributed matrix multiplication to Spark. We document an efficient distributed matrix multiplication using Cannon’s algorithm, which improves significantly on the performance of the existing MLlib implementation. Used within a barrier task, the algorithm described herein results in an up to 24% performance increase on a 10,000x10,000 square matrix with a significantly lower memory footprint. Applications of efficient matrix multiplication include, among others, accelerating the training and implementation of deep convolutional neural network based workloads, and thus such efficient algorithms can play a ground-breaking role in faster, more efficient execution of even the most complicated machine learning tasks

scholarly journals Encoding Health Records into Pathway Representations for Deep Learning

2021 ◽  
Author(s):  
Marco Luca Sbodio ◽  
Natasha Mulligan ◽  
Stefanie Speichert ◽  
Vanessa Lopez ◽  
Joao Bettencourt-Silva
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Deep Learning ◽  
Source Code ◽  
Training Dataset ◽  
Health Records ◽  
Learning Tasks ◽  
Patient Pathways ◽  
Computational Resources ◽  
The Impact

There is a growing trend in building deep learning patient representations from health records to obtain a comprehensive view of a patient’s data for machine learning tasks. This paper proposes a reproducible approach to generate patient pathways from health records and to transform them into a machine-processable image-like structure useful for deep learning tasks. Based on this approach, we generated over a million pathways from FAIR synthetic health records and used them to train a convolutional neural network. Our initial experiments show the accuracy of the CNN on a prediction task is comparable or better than other autoencoders trained on the same data, while requiring significantly less computational resources for training. We also assess the impact of the size of the training dataset on autoencoders performances. The source code for generating pathways from health records is provided as open source.

scholarly journals PyDL8.5: a Library for Learning Optimal Decision Trees

2020 ◽  
Author(s):  
Gaël Aglin ◽  
Siegfried Nijssen ◽  
Pierre Schaus
Keyword(s):  
Machine Learning ◽  
Decision Trees ◽  
Efficient Algorithm ◽  
Optimal Decision ◽  
Learning Tasks ◽  
Explainable Ai ◽  
Classification Tasks ◽  
Interpretable Models ◽  
Limited Depth

Decision Trees (DTs) are widely used Machine Learning (ML) models with a broad range of applications. The interest in these models has increased even further in the context of Explainable AI (XAI), as decision trees of limited depth are very interpretable models. However, traditional algorithms for learning DTs are heuristic in nature; they may produce trees that are of suboptimal quality under depth constraints. We introduce PyDL8.5, a Python library to infer depth-constrained Optimal Decision Trees (ODTs). PyDL8.5 provides an interface for DL8.5, an efficient algorithm for inferring depth-constrained ODTs. The library provides an easy-to-use scikit-learn compatible interface. It cannot only be used for classification tasks, but also for regression, clustering, and other tasks. We introduce an interface that allows users to easily implement these other learning tasks. We provide a number of examples of how to use this library.

Building scalable variational circuit training for machine learning tasks

2021 ◽  
Author(s):  
Kathleen E. Hamilton ◽  
Emily Lynn ◽  
Tyler Kharazi ◽  
Titus Morris ◽  
Ryan S. Bennink ◽  
...  
Keyword(s):  
Machine Learning ◽  
Circuit Training ◽  
Learning Tasks
Sign in / Sign up

Export Citation Format

Share Document