scholarly journals The use of deep learning to automate the segmentation of the skeleton from CT volumes of pigs1

2018 ◽  
Vol 2 (3) ◽  
pp. 324-335 ◽  
Author(s):  
Johannes Kvam ◽  
Lars Erik Gangsei ◽  
Jørgen Kongsro ◽  
Anne H Schistad Solberg
Keyword(s):  
Deep Learning ◽  
Automatic Segmentation ◽  
Large Data ◽  
Ct Scanning ◽  
Large Data Sets ◽  
Training Data ◽  
Data Sets ◽  
Fully Automatic ◽  
Number Of Individuals ◽  
Control Accuracy

Abstract Computed tomography (CT) scanning of pigs has been shown to produce detailed phenotypes useful in pig breeding. Due to the large number of individuals scanned and corresponding large data sets, there is a need for automatic tools for analysis of these data sets. In this paper, the feasibility of deep learning for fully automatic segmentation of the skeleton of pigs from CT volumes is explored. To maximize performance, given the training data available, a series of problem simplifications are applied. The deep-learning approach can replace our currently used semiautomatic solution, with increased robustness and little or no need for manual control. Accuracy was highly affected by training data, and expanding the training set can further increase performance making this approach especially promising.

scholarly journals Online Deep Learning: Learning Deep Neural Networks on the Fly

2018 ◽  
Author(s):  
Doyen Sahoo ◽  
Quang Pham ◽  
Jing Lu ◽  
Steven C. H. Hoi
Keyword(s):  
Neural Networks ◽  
Online Learning ◽  
Deep Learning ◽  
Deep Neural Networks ◽  
Large Data ◽  
Learning Task ◽  
Large Data Sets ◽  
Training Data ◽  
Data Sets ◽  
Online Setting

Deep Neural Networks (DNNs) are typically trained by backpropagation in a batch setting, requiring the entire training data to be made available prior to the learning task. This is not scalable for many real-world scenarios where new data arrives sequentially in a stream. We aim to address an open challenge of ``Online Deep Learning" (ODL) for learning DNNs on the fly in an online setting. Unlike traditional online learning that often optimizes some convex objective function with respect to a shallow model (e.g., a linear/kernel-based hypothesis), ODL is more challenging as the optimization objective is non-convex, and regular DNN with standard backpropagation does not work well in practice for online settings. We present a new ODL framework that attempts to tackle the challenges by learning DNN models which dynamically adapt depth from a sequence of training data in an online learning setting. Specifically, we propose a novel Hedge Backpropagation (HBP) method for online updating the parameters of DNN effectively, and validate the efficacy on large data sets (both stationary and concept drifting scenarios).

Deep Learning Approaches for Sentiment Analysis Challenges and Future Issues

2022 ◽  
pp. 27-50
Author(s):  
Rajalaxmi Prabhu B. ◽  
Seema S.
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Model Building ◽  
Large Data ◽  
Machine Learning Algorithms ◽  
Large Data Sets ◽  
Data Sets ◽  
Learning Approaches ◽  
Learning Techniques ◽  
Important Challenge

A lot of user-generated data is available these days from huge platforms, blogs, websites, and other review sites. These data are usually unstructured. Analyzing sentiments from these data automatically is considered an important challenge. Several machine learning algorithms are implemented to check the opinions from large data sets. A lot of research has been undergone in understanding machine learning approaches to analyze sentiments. Machine learning mainly depends on the data required for model building, and hence, suitable feature exactions techniques also need to be carried. In this chapter, several deep learning approaches, its challenges, and future issues will be addressed. Deep learning techniques are considered important in predicting the sentiments of users. This chapter aims to analyze the deep-learning techniques for predicting sentiments and understanding the importance of several approaches for mining opinions and determining sentiment polarity.

scholarly journals An Improved Fellegi-Sunter Framework for Probabilistic Record Linkage Between Large Data Sets

2020 ◽  
Vol 36 (4) ◽  
pp. 803-825
Author(s):  
Marco Fortini
Keyword(s):  
Record Linkage ◽  
Large Data ◽  
Real Data ◽  
Large Data Sets ◽  
Training Data ◽  
Data Sets ◽  
Estimated Parameters ◽  
Unbiased Estimates ◽  
Structural Zeros ◽  
Different Sources

AbstractRecord linkage addresses the problem of identifying pairs of records coming from different sources and referred to the same unit of interest. Fellegi and Sunter propose an optimal statistical test in order to assign the match status to the candidate pairs, in which the needed parameters are obtained through EM algorithm directly applied to the set of candidate pairs, without recourse to training data. However, this procedure has a quadratic complexity as the two lists to be matched grow. In addition, a large bias of EM-estimated parameters is also produced in this case, so that the problem is tackled by reducing the set of candidate pairs through filtering methods such as blocking. Unfortunately, the probability that excluded pairs would be actually true-matches cannot be assessed through such methods.The present work proposes an efficient approach in which the comparison of records between lists are minimised while the EM estimates are modified by modelling tables with structural zeros in order to obtain unbiased estimates of the parameters. Improvement achieved by the suggested method is shown by means of simulations and an application based on real data.

On the Distribution of Species Occurrence

Paleobiology ◽  
1982 ◽  
Vol 8 (2) ◽  
pp. 143-150 ◽  
Author(s):  
Martin A. Buzas ◽  
Carl F. Koch ◽  
Stephen J. Culver ◽  
Norman F. Sohl
Keyword(s):  
Gulf Coast ◽  
Species Abundance ◽  
Large Data ◽  
Abundant Species ◽  
Large Data Sets ◽  
Data Sets ◽  
Species Occurrence ◽  
Extreme Caution ◽  
Lognormal Distributions ◽  
Number Of Individuals

The distribution of species abundance (number of individuals per species) is well documented. The distribution of species occurrence (number of localities per species), however, has received little attention. This study investigates the distribution of species occurrence for five large data sets. For modern benthic foraminifera, species occurrence is examined from the Atlantic continental margin of North America, where 875 species were recorded 10,017 times at 542 localities, the Gulf of Mexico, where 848 species were recorded 18,007 times at 426 localities, and the Caribbean, where 1149 species were recorded 6684 times at 268 localities. For Late Cretaceous molluscs, species occurrence is examined from the Gulf Coast where 716 species were recorded 6236 times at 166 localities and a subset of this data consisting of 643 species recorded 3851 times at 86 localities.Logseries and lognormal distributions were fitted to these data sets. In most instances the logseries best predicts the distribution of species occurrence. The lognormal, however, also fits the data fairly well, and, in one instance, better. The use of these distributions allows the prediction of the number of species occurring once, twice, …, n times.Species abundance data are also available for the molluscan data sets. They indicate that the most abundant species (greatest number of individuals) usually occur most frequently. In all data sets approximately half the species occur four or less times. The probability of noting the presence of rarely occurring species is small, and, consequently, such species must be used with extreme caution in studies requiring knowledge of the distribution of species in space and time.

An Investigation Into the Efficacy of Deep Learning Tools for Big Data Analysis in Health Care

2018 ◽  
Vol 10 (3) ◽  
pp. 1-13 ◽  
Author(s):  
Rojalina Priyadarshini ◽  
Rabindra K. Barik ◽  
Chhabi Panigrahi ◽  
Harishchandra Dubey ◽  
Brojo Kishore Mishra
Keyword(s):  
Big Data ◽  
Deep Learning ◽  
Large Data ◽  
Optimization Techniques ◽  
Large Data Sets ◽  
Data Sets ◽  
Learning Tools ◽  
Healthcare Applications ◽  
Proper Training ◽  
Future Prediction

This article describes how machine learning (ML) algorithms are very useful for analysis of data and finding some meaningful information out of them, which could be used in various other applications. In the last few years, an explosive growth has been seen in the dimension and structure of data. There are several difficulties faced by conventional ML algorithms while dealing with such highly voluminous and unstructured big data. The modern ML tools are designed and used to deal with all sorts of complexities of data. Deep learning (DL) is one of the modern ML tools which are commonly used to find the hidden structure and cohesion among these large data sets by giving proper training in parallel platforms with intelligent optimization techniques to further analyze and interpret the data for future prediction and classification. This article focuses on the use of DL tools and software which are used in past couple of years in various areas and especially in the area of healthcare applications.

scholarly journals Car-Following Described by Blending Data-Driven and Analytical Models: A Gaussian Process Regression Approach

2021 ◽  
pp. 036119812110326
Author(s):  
Ignasi Echaniz Soldevila ◽  
Victor L. Knoop ◽  
Serge Hoogendoorn
Keyword(s):  
Gaussian Process Regression ◽  
Large Data ◽  
Driving Behavior ◽  
Large Data Sets ◽  
Training Data ◽  
Data Driven ◽  
Data Sets ◽  
Data Set ◽  
Car Following ◽  
New Variables

Traffic engineers rely on microscopic traffic models to design, plan, and operate a wide range of traffic applications. Recently, large data sets, yet incomplete and from small space regions, are becoming available thanks to technology improvements and governmental efforts. With this study we aim to gain new empirical insights into longitudinal driving behavior and to formulate a model which can benefit from these new challenging data sources. This paper proposes an application of an existing formulation, Gaussian process regression (GPR), to describe individual longitudinal driving behavior of drivers. The method integrates a parametric and a non-parametric mathematical formulation. The model predicts individual driver’s acceleration given a set of variables. It uses the GPR to make predictions when there exists correlation between new input and the training data set. The data-driven model benefits from a large training data set to capture all driver longitudinal behavior, which would be difficult to fit in fixed parametric equation(s). The methodology allows us to train models with new variables without the need of altering the model formulation. And importantly, the model also uses existing traditional parametric car-following models to predict acceleration when no similar situations are found in the training data set. A case study using radar data in an urban environment shows that a hybrid model performs better than parametric model alone and suggests that traffic light status over time influences drivers’ acceleration. This methodology can help engineers to use large data sets and to find new variables to describe traffic behavior.

scholarly journals A convolutional neural network-based screening tool for X-ray serial crystallography

2018 ◽  
Vol 25 (3) ◽  
pp. 655-670 ◽  
Author(s):  
Tsung-Wei Ke ◽  
Aaron S. Brewster ◽  
Stella X. Yu ◽  
Daniela Ushizima ◽  
Chao Yang ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Large Data ◽  
Large Data Sets ◽  
Training Data ◽  
Data Sets ◽  
X Ray ◽  
X Ray Crystallography ◽  
Automatic Image Processing

A new tool is introduced for screening macromolecular X-ray crystallography diffraction images produced at an X-ray free-electron laser light source. Based on a data-driven deep learning approach, the proposed tool executes a convolutional neural network to detect Bragg spots. Automatic image processing algorithms described can enable the classification of large data sets, acquired under realistic conditions consisting of noisy data with experimental artifacts. Outcomes are compared for different data regimes, including samples from multiple instruments and differing amounts of training data for neural network optimization.

A Practical Robust and Efficient RBF Metamodel Method for Typical Engineering Problems

2008 ◽  
Author(s):  
Xingjie Fang ◽  
Liping Wang ◽  
Don Beeson ◽  
Gene Wiggs
Keyword(s):  
Principal Component ◽  
Large Data ◽  
Large Data Sets ◽  
Training Data ◽  
Data Sets ◽  
Dimensional Model ◽  
Data Set ◽  
Engineering Problems ◽  
Processing Techniques ◽  
Generalization Accuracy

Radial Basis Function (RBF) metamodels have recently attracted increased interest due to their significant advantages over other types of non-parametric metamodels. However, because of the interpolation nature of the RBF mathematics, the accuracy of the model may dramatically deteriorate if the training data set used contains duplicate information, noise or outliers. Also constructing the metamodel may be time consuming whenever the training data sets are large or a high dimensional model is required. In this paper, we propose a robust and efficient RBF metamodeling approach based on data pre-processing techniques that alleviate the accuracy and efficiency issues commonly encountered when RBF models are used in typical real engineering situations. These techniques include 1) the removal of duplicate training data information, 2) the generation of smaller uniformly distributed subsets of training data from large data sets and 3) the quantification and identification of outliers by principal component analysis (PCA) and Hotelling statistics. Simulation results are used to validate the generalization accuracy and efficiency of the proposed approach.

scholarly journals Deep Learning Security Systems

2019 ◽  
Vol 8 (6S3) ◽  
pp. 1823-1826
Keyword(s):  
Big Data ◽  
Deep Learning ◽  
Data Analysis ◽  
Data Analytics ◽  
Big Data Analytics ◽  
Large Data ◽  
Large Data Sets ◽  
Data Sets ◽  
Security Systems ◽  
Abstract Knowledge

Big Data Analytics and Deep Learning are not supposed to be two entirely different concepts. Big Data means extremely huge large data sets that can be analyzed to find patterns, trends. One technique that can be used for data analysis so that able to help us find abstract patterns in Big Data is Deep Learning. If we apply Deep Learning to Big Data, we can find unknown and useful patterns that were impossible so far. With the help of Deep Learning, AI is getting smart. There is a hypothesis in this regard, the more data, the more abstract knowledge. So a handy survey of Big Data, Deep Learning and its application in Big Data is necessary.

Sign in / Sign up

Export Citation Format

Share Document