Extend Mixed Models to Multi-layer Neural Networks for Genomic Prediction Including Intermediate Omics Data

ABSTRACTWith the growing amount and diversity of intermediate omics data complementary to genomics (e.g., DNA methylation, gene expression, and protein abundance), there is a need to develop methods to incorporate intermediate omics data into conventional genomic evaluation. The omics data helps decode the multiple layers of regulation from genotypes to phenotypes, thus forms a connected multi-layer network naturally. We developed a new method named NN-LMM to model the multiple layers of regulation from genotypes to intermediate omics features, then to phenotypes, by extending conventional linear mixed models (“LMM”) to multi-layer artificial neural networks (“NN”). NN-LMM incorporates intermediate omics features by adding middle layers between genotypes and phenotypes. Linear mixed models (e.g., pedigree-based BLUP, GBLUP, Bayesian Alphabet, single-step GBLUP, or single-step Bayesian Alphabet) can be used to sample marker effects or genetic values on intermediate omics features, and activation functions in neural networks are used to capture the nonlinear relationships between intermediate omics features and phenotypes. NN-LMM had significantly better prediction performance than the recently proposed single-step approach for genomic prediction with intermediate omics data. Compared to the single-step approach, NN-LMM can handle various patterns of missing omics measures, and allows nonlinear relationships between intermediate omics features and phenotypes. NN-LMM has been implemented in an open-source package called “JWAS”.

Download Full-text

Genomic prediction for crossbred performance using metafounders1

Journal of Animal Science ◽

10.1093/jas/sky433 ◽

2018 ◽

Vol 97 (2) ◽

pp. 548-558 ◽

Cited By ~ 3

Author(s):

Elizabeth M van Grevenhof ◽

Jérémie Vandenplas ◽

Mario P L Calus

Keyword(s):

Genomic Prediction ◽

Variance Components ◽

Single Step ◽

Convergence Properties ◽

Genomic Evaluation ◽

Linear Unbiased Prediction ◽

Breeding Programs ◽

Genetic Groups ◽

Best Linear Unbiased ◽

Definition Of

Abstract Future genomic evaluation models to be used routinely in breeding programs for pigs and poultry need to be able to optimally use information of crossbred (CB) animals to predict breeding values for CB performance of purebred (PB) selection candidates. Important challenges in the commonly used single-step genomic best linear unbiased prediction (ssGBLUP) model are the definition of relationships between the different line compositions and the definition of the base generation per line. The use of metafounders (MFs) in ssGBLUP has been proposed to overcome these issues. When relationships between lines are known to be different from 0, the use of MFs generalizes the concept of genetic groups relying on the genotype data. Our objective was to investigate the effect of using MFs in genomic prediction for CB performance on estimated variance components, and accuracy and bias of GEBV. This was studied using stochastic simulation to generate data representing a three-way crossbreeding scheme in pigs, with the parental lines being either closely related or unrelated. Results show that using MFs, the variance components should be scaled appropriately, especially when basing them on estimates obtained with, for example a pedigree-based model. The accuracies of GEBV that were obtained using MFs were similar to accuracies without using MFs, regardless whether the lines involved in the CB were closely related or unrelated. The use of MFs resulted in a model that had similar or somewhat better convergence properties compared to other models. We recommend the use of MFs in ssGBLUP for genomic evaluations in crossbreeding schemes.

Download Full-text

Back-propagation neural networks and generalized linear mixed models to investigate vehicular flow and weather data relationships with crash severity in urban road segments

Transport Infrastructure and Systems ◽

10.1201/9781315281896-94 ◽

2017 ◽

pp. 731-740 ◽

Cited By ~ 1

Author(s):

L. Mussone ◽

M. Bassani ◽

P. Masci

Keyword(s):

Neural Networks ◽

Mixed Models ◽

Generalized Linear Mixed Models ◽

Linear Mixed Models ◽

Back Propagation ◽

Weather Data ◽

Crash Severity ◽

Urban Road ◽

Back Propagation Neural Networks ◽

Road Segments

Download Full-text

Analysis of Non-Linear Activation Functions for Classification Tasks Using Convolutional Neural Networks

Recent Patents on Computer Science ◽

10.2174/2213275911666181025143029 ◽

2019 ◽

Vol 12 (3) ◽

pp. 156-161 ◽

Cited By ~ 3

Author(s):

Aman Dureja ◽

Payal Pahwa

Keyword(s):

Neural Networks ◽

Deep Neural Networks ◽

Activation Function ◽

Primary Objective ◽

Experimental Comparison ◽

Activation Functions ◽

Practical Applications ◽

Network Activation ◽

Non Linear ◽

Hidden Layer

Background: In making the deep neural network, activation functions play an important role. But the choice of activation functions also affects the network in term of optimization and to retrieve the better results. Several activation functions have been introduced in machine learning for many practical applications. But which activation function should use at hidden layer of deep neural networks was not identified. Objective: The primary objective of this analysis was to describe which activation function must be used at hidden layers for deep neural networks to solve complex non-linear problems. Methods: The configuration for this comparative model was used by using the datasets of 2 classes (Cat/Dog). The number of Convolutional layer used in this network was 3 and the pooling layer was also introduced after each layer of CNN layer. The total of the dataset was divided into the two parts. The first 8000 images were mainly used for training the network and the next 2000 images were used for testing the network. Results: The experimental comparison was done by analyzing the network by taking different activation functions on each layer of CNN network. The validation error and accuracy on Cat/Dog dataset were analyzed using activation functions (ReLU, Tanh, Selu, PRelu, Elu) at number of hidden layers. Overall the Relu gave best performance with the validation loss at 25th Epoch 0.3912 and validation accuracy at 25th Epoch 0.8320. Conclusion: It is found that a CNN model with ReLU hidden layers (3 hidden layers here) gives best results and improve overall performance better in term of accuracy and speed. These advantages of ReLU in CNN at number of hidden layers are helpful to effectively and fast retrieval of images from the databases.

Download Full-text

Hardware implementation of radial-basis neural networks with Gaussian activation functions on FPGA

Neural Computing and Applications ◽

10.1007/s00521-021-05706-3 ◽

2021 ◽

Author(s):

Volodymyr Shymkovych ◽

Sergii Telenyk ◽

Petro Kravets

Keyword(s):

Neural Networks ◽

Hardware Implementation ◽

Gaussian Function ◽

Activation Function ◽

Rbf Neural Networks ◽

Activation Functions ◽

Rbf Network ◽

Combination Scheme ◽

Radial Basis ◽

Hidden Layer

AbstractThis article introduces a method for realizing the Gaussian activation function of radial-basis (RBF) neural networks with their hardware implementation on field-programmable gaits area (FPGAs). The results of modeling of the Gaussian function on FPGA chips of different families have been presented. RBF neural networks of various topologies have been synthesized and investigated. The hardware component implemented by this algorithm is an RBF neural network with four neurons of the latent layer and one neuron with a sigmoid activation function on an FPGA using 16-bit numbers with a fixed point, which took 1193 logic matrix gate (LUTs—LookUpTable). Each hidden layer neuron of the RBF network is designed on an FPGA as a separate computing unit. The speed as a total delay of the combination scheme of the block RBF network was 101.579 ns. The implementation of the Gaussian activation functions of the hidden layer of the RBF network occupies 106 LUTs, and the speed of the Gaussian activation functions is 29.33 ns. The absolute error is ± 0.005. The Spartan 3 family of chips for modeling has been used to get these results. Modeling on chips of other series has been also introduced in the article. RBF neural networks of various topologies have been synthesized and investigated. Hardware implementation of RBF neural networks with such speed allows them to be used in real-time control systems for high-speed objects.

Download Full-text

Trigonometric Inference Providing Learning in Deep Neural Networks

Applied Sciences ◽

10.3390/app11156704 ◽

2021 ◽

Vol 11 (15) ◽

pp. 6704

Author(s):

Jingyong Cai ◽

Masashi Takemoto ◽

Yuming Qiu ◽

Hironori Nakajo

Keyword(s):

Machine Learning ◽

Neural Networks ◽

Deep Neural Networks ◽

Activation Function ◽

Trigonometric Approximation ◽

Model Parameters ◽

Training Algorithms ◽

Activation Functions ◽

Classical Training ◽

Sum Formula

Despite being heavily used in the training of deep neural networks (DNNs), multipliers are resource-intensive and insufficient in many different scenarios. Previous discoveries have revealed the superiority when activation functions, such as the sigmoid, are calculated by shift-and-add operations, although they fail to remove multiplications in training altogether. In this paper, we propose an innovative approach that can convert all multiplications in the forward and backward inferences of DNNs into shift-and-add operations. Because the model parameters and backpropagated errors of a large DNN model are typically clustered around zero, these values can be approximated by their sine values. Multiplications between the weights and error signals are transferred to multiplications of their sine values, which are replaceable with simpler operations with the help of the product to sum formula. In addition, a rectified sine activation function is utilized for further converting layer inputs into sine values. In this way, the original multiplication-intensive operations can be computed through simple add-and-shift operations. This trigonometric approximation method provides an efficient training and inference alternative for devices with insufficient hardware multipliers. Experimental results demonstrate that this method is able to obtain a performance close to that of classical training algorithms. The approach we propose sheds new light on future hardware customization research for machine learning.

Download Full-text

Heat, Hills and the High Season: A Model-Based Comparative Analysis of Spatio-Temporal Factors Affecting Shared Bicycle Use in Three Southern European Islands

Sustainability ◽

10.3390/su13063274 ◽

2021 ◽

Vol 13 (6) ◽

pp. 3274

Author(s):

Suzanne Maas ◽

Paraskevas Nikolaou ◽

Maria Attard ◽

Loukas Dimitriou

Keyword(s):

Correlation Analysis ◽

Mixed Models ◽

Linear Mixed Models ◽

Positive Association ◽

Sustainable Mobility ◽

Weather Factors ◽

Gran Canaria ◽

Bivariate Correlation ◽

Temporal Factors

Bicycle sharing systems (BSSs) have been implemented in cities worldwide in an attempt to promote cycling. Despite exhibiting characteristics considered to be barriers to cycling, such as hot summers, hilliness and car-oriented infrastructure, Southern European island cities and tourist destinations Limassol (Cyprus), Las Palmas de Gran Canaria (Canary Islands, Spain) and the Valletta conurbation (Malta) are all experiencing the implementation of BSSs and policies to promote cycling. In this study, a year of trip data and secondary datasets are used to analyze dock-based BSS usage in the three case-study cities. How land use, socio-economic, network and temporal factors influence BSS use at station locations, both as an origin and as a destination, was examined using bivariate correlation analysis and through the development of linear mixed models for each case study. Bivariate correlations showed significant positive associations with the number of cafes and restaurants, vicinity to the beach or promenade and the percentage of foreign population at the BSS station locations in all cities. A positive relation with cycling infrastructure was evident in Limassol and Las Palmas de Gran Canaria, but not in Malta, as no cycling infrastructure is present in the island’s conurbation, where the BSS is primarily operational. Elevation had a negative association with BSS use in all three cities. In Limassol and Malta, where seasonality in weather patterns is strongest, a negative effect of rainfall and a positive effect of higher temperature were observed. Although there was a positive association between BSS use and the number of visiting tourists in Limassol and Malta, this is predominantly explained through the multi-collinearity with weather factors rather than by intensive use of the BSS by tourists. The linear mixed models showed more fine-grained results and explained differences in BSS use at stations, including differences for station use as an origin and as a destination. The insights from the correlation analysis and linear mixed models can be used to inform policies promoting cycling and BSS use and support sustainable mobility policies in the case-study cities and cities with similar characteristics.

Download Full-text

Genomic Prediction Using Alternative Strategies of Weighted Single-Step Genomic BLUP for Yearling Weight and Carcass Traits in Hanwoo Beef Cattle

Genes ◽

10.3390/genes12020266 ◽

2021 ◽

Vol 12 (2) ◽

pp. 266

Author(s):

Hossein Mehrban ◽

Masoumeh Naserkheil ◽

Deuk Hwan Lee ◽

Chungil Cho ◽

Taejeong Choi ◽

...

Keyword(s):

Quantitative Trait Loci ◽

Beef Cattle ◽

Genomic Prediction ◽

Quantitative Trait ◽

Carcass Traits ◽

Best Linear Unbiased Prediction ◽

Single Step ◽

Linear Unbiased Prediction ◽

Single Nucleotide ◽

Best Linear Unbiased

The weighted single-step genomic best linear unbiased prediction (GBLUP) method has been proposed to exploit information from genotyped and non-genotyped relatives, allowing the use of weights for single-nucleotide polymorphism in the construction of the genomic relationship matrix. The purpose of this study was to investigate the accuracy of genetic prediction using the following single-trait best linear unbiased prediction methods in Hanwoo beef cattle: pedigree-based (PBLUP), un-weighted (ssGBLUP), and weighted (WssGBLUP) single-step genomic methods. We also assessed the impact of alternative single and window weighting methods according to their effects on the traits of interest. The data was comprised of 15,796 phenotypic records for yearling weight (YW) and 5622 records for carcass traits (backfat thickness: BFT, carcass weight: CW, eye muscle area: EMA, and marbling score: MS). Also, the genotypic data included 6616 animals for YW and 5134 for carcass traits on the 43,950 single-nucleotide polymorphisms. The ssGBLUP showed significant improvement in genomic prediction accuracy for carcass traits (71%) and yearling weight (99%) compared to the pedigree-based method. The window weighting procedures performed better than single SNP weighting for CW (11%), EMA (11%), MS (3%), and YW (6%), whereas no gain in accuracy was observed for BFT. Besides, the improvement in accuracy between window WssGBLUP and the un-weighted method was low for BFT and MS, while for CW, EMA, and YW resulted in a gain of 22%, 15%, and 20%, respectively, which indicates the presence of relevant quantitative trait loci for these traits. These findings indicate that WssGBLUP is an appropriate method for traits with a large quantitative trait loci effect.

Download Full-text