scholarly journals Computational Complexity Reduction of Neural Networks of Brain Tumor Image Segmentation by Introducing Fermi–Dirac Correction Functions

Entropy ◽  
2021 ◽  
Vol 23 (2) ◽  
pp. 223
Author(s):  
Yen-Ling Tai ◽  
Shin-Jhe Huang ◽  
Chien-Chang Chen ◽  
Henry Horng-Shing Lu
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Computational Complexity ◽  
High Performance ◽  
Low Cost ◽  
Structural Complexity ◽  
Correction Function ◽  
Computational Time ◽  
Learning Methods ◽  
Band Theory

Nowadays, deep learning methods with high structural complexity and flexibility inevitably lean on the computational capability of the hardware. A platform with high-performance GPUs and large amounts of memory could support neural networks having large numbers of layers and kernels. However, naively pursuing high-cost hardware would probably drag the technical development of deep learning methods. In the article, we thus establish a new preprocessing method to reduce the computational complexity of the neural networks. Inspired by the band theory of solids in physics, we map the image space into a noninteraction physical system isomorphically and then treat image voxels as particle-like clusters. Then, we reconstruct the Fermi–Dirac distribution to be a correction function for the normalization of the voxel intensity and as a filter of insignificant cluster components. The filtered clusters at the circumstance can delineate the morphological heterogeneity of the image voxels. We used the BraTS 2019 datasets and the dimensional fusion U-net for the algorithmic validation, and the proposed Fermi–Dirac correction function exhibited comparable performance to other employed preprocessing methods. By comparing to the conventional z-score normalization function and the Gamma correction function, the proposed algorithm can save at least 38% of computational time cost under a low-cost hardware architecture. Even though the correction function of global histogram equalization has the lowest computational time among the employed correction functions, the proposed Fermi–Dirac correction function exhibits better capabilities of image augmentation and segmentation.

Wavelets as activation functions in Neural Networks

2021 ◽  
pp. 1-11
Author(s):  
Oscar Herrera ◽  
Belém Priego
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
High Performance ◽  
Research Area ◽  
Experimental Results ◽  
Activation Functions ◽  
Hyperbolic Tangent ◽  
Open Research ◽  
Bounded Functions

Traditionally, a few activation functions have been considered in neural networks, including bounded functions such as threshold, sigmoidal and hyperbolic-tangent, as well as unbounded ReLU, GELU, and Soft-plus, among other functions for deep learning, but the search for new activation functions still being an open research area. In this paper, wavelets are reconsidered as activation functions in neural networks and the performance of Gaussian family wavelets (first, second and third derivatives) are studied together with other functions available in Keras-Tensorflow. Experimental results show how the combination of these activation functions can improve the performance and supports the idea of extending the list of activation functions to wavelets which can be available in high performance platforms.

Human Skin Detection in Color Images Using Deep Learning

2015 ◽  
Vol 5 (2) ◽  
pp. 1-13 ◽  
Author(s):  
Mohammadreza Hajiarbabi ◽  
Arvin Agah
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Human Skin ◽  
Skin Color ◽  
Color Image ◽  
Gaussian Model ◽  
Color Images ◽  
Skin Detection ◽  
Learning Methods ◽  
Rule Based

Human skin detection is an important and challenging problem in computer vision. Skin detection can be used as the first phase in face detection when using color images. The differences in illumination and ranges of skin colors have made skin detection a challenging task. Gaussian model, rule based methods, and artificial neural networks are methods that have been used for human skin color detection. Deep learning methods are new techniques in learning that have shown improved classification power compared to neural networks. In this paper the authors use deep learning methods in order to enhance the capabilities of skin detection algorithms. Several experiments have been performed using auto encoders and different color spaces. The proposed technique is evaluated compare with other available methods in this domain using two color image databases. The results show that skin detection utilizing deep learning has better results compared to other methods such as rule-based, Gaussian model and feed forward neural network.

scholarly journals Nesting Monte Carlo for high-dimensional non-linear PDEs

2018 ◽  
Vol 24 (4) ◽  
pp. 225-247 ◽  
Author(s):  
Xavier Warin
Keyword(s):  
Monte Carlo ◽  
Deep Learning ◽  
High Dimension ◽  
Analytical Solutions ◽  
New Method ◽  
Computational Time ◽  
High Dimensional ◽  
Learning Methods ◽  
Lipschitz Constants ◽  
Linear Pdes

Abstract A new method based on nesting Monte Carlo is developed to solve high-dimensional semi-linear PDEs. Depending on the type of non-linearity, different schemes are proposed and theoretically studied: variance error are given and it is shown that the bias of the schemes can be controlled. The limitation of the method is that the maturity or the Lipschitz constants of the non-linearity should not be too high in order to avoid an explosion of the computational time. Many numerical results are given in high dimension for cases where analytical solutions are available or where some solutions can be computed by deep-learning methods.

scholarly journals The Old and the New: Can Physics-Informed Deep-Learning Replace Traditional Linear Solvers?

2021 ◽  
Vol 4 ◽  
Author(s):  
Stefano Markidis
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
High Performance ◽  
Critical Role ◽  
Low Frequency ◽  
Accurate Solution ◽  
Limiting Factor ◽  
Data Set ◽  
Linear Solvers ◽  
Computational Performance

Physics-Informed Neural Networks (PINN) are neural networks encoding the problem governing equations, such as Partial Differential Equations (PDE), as a part of the neural network. PINNs have emerged as a new essential tool to solve various challenging problems, including computing linear systems arising from PDEs, a task for which several traditional methods exist. In this work, we focus first on evaluating the potential of PINNs as linear solvers in the case of the Poisson equation, an omnipresent equation in scientific computing. We characterize PINN linear solvers in terms of accuracy and performance under different network configurations (depth, activation functions, input data set distribution). We highlight the critical role of transfer learning. Our results show that low-frequency components of the solution converge quickly as an effect of the F-principle. In contrast, an accurate solution of the high frequencies requires an exceedingly long time. To address this limitation, we propose integrating PINNs into traditional linear solvers. We show that this integration leads to the development of new solvers whose performance is on par with other high-performance solvers, such as PETSc conjugate gradient linear solvers, in terms of performance and accuracy. Overall, while the accuracy and computational performance are still a limiting factor for the direct use of PINN linear solvers, hybrid strategies combining old traditional linear solver approaches with new emerging deep-learning techniques are among the most promising methods for developing a new class of linear solvers.

Using AI at the Edge and Incremental Machine Learning to Process Onboard Instrument Data

2021 ◽  
Author(s):  
Nicholas Parkyn
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Real Time ◽  
Data Storage ◽  
High Performance ◽  
Heterogeneous Computing ◽  
Low Cost ◽  
Machine Intelligence ◽  
Edge Computing ◽  
Continual Learning

Emerging heterogeneous computing, computing at the edge, machine learning and AI at the edge technology drives approaches and techniques for processing and analysing onboard instrument data in near real-time. The author has used edge computing and neural networks combined with high performance heterogeneous computing platforms to accelerate AI workloads. Heterogeneous computing hardware used is readily available, low cost, delivers impressive AI performance and can run multiple neural networks in parallel. Collecting, processing and machine learning from onboard instruments data in near real-time is not a trivial problem due to data volumes, complexities of data filtering, data storage and continual learning. Little research has been done on continual machine learning which aims at a higher level of machine intelligence through providing the artificial agents with the ability to learn from a non-stationary and never-ending stream of data. The author has applied the concept of continual learning to building a system that continually learns from actual boat performance and refines predictions previously done using static VPP data. The neural networks used are initially trained using the output from traditional VPP software and continue to learn from actual data collected under real sailing conditions. The author will present the system design, AI, and edge computing techniques used and the approaches he has researched for incremental training to realise continual learning.

scholarly journals Neural network activation similarity: a new measure to assist decision making in chemical toxicology

2020 ◽  
Vol 11 (28) ◽  
pp. 7335-7348 ◽  
Author(s):  
Timothy E. H. Allen ◽  
Andrew J. Wedlake ◽  
Elena Gelžinytė ◽  
Charles Gong ◽  
Jonathan M. Goodman ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Decision Making ◽  
Deep Learning ◽  
Test Data ◽  
High Performance ◽  
Chemical Binding ◽  
Biological Targets ◽  
Network Activation ◽  
Chemical Toxicology

Deep learning neural networks, constructed for the prediction of chemical binding at 79 pharmacologically important human biological targets, show extremely high performance on test data (accuracy 92.2 ± 4.2%, MCC 0.814 ± 0.093, ROC-AUC 0.96 ± 0.04).

scholarly journals An Investigation of Deep Learning Models for EEG-Based Emotion Recognition

2020 ◽  
Vol 14 ◽  
Author(s):  
Yaqing Zhang ◽  
Jinling Chen ◽  
Jen Hong Tan ◽  
Yuxuan Chen ◽  
Yunyi Chen ◽  
...  
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Emotion Recognition ◽  
Real Life ◽  
Learning Rate ◽  
Learning Models ◽  
Learning Methods ◽  
Time Frequency ◽  
Comparison Results

Emotion is the human brain reacting to objective things. In real life, human emotions are complex and changeable, so research into emotion recognition is of great significance in real life applications. Recently, many deep learning and machine learning methods have been widely applied in emotion recognition based on EEG signals. However, the traditional machine learning method has a major disadvantage in that the feature extraction process is usually cumbersome, which relies heavily on human experts. Then, end-to-end deep learning methods emerged as an effective method to address this disadvantage with the help of raw signal features and time-frequency spectrums. Here, we investigated the application of several deep learning models to the research field of EEG-based emotion recognition, including deep neural networks (DNN), convolutional neural networks (CNN), long short-term memory (LSTM), and a hybrid model of CNN and LSTM (CNN-LSTM). The experiments were carried on the well-known DEAP dataset. Experimental results show that the CNN and CNN-LSTM models had high classification performance in EEG-based emotion recognition, and their accurate extraction rate of RAW data reached 90.12 and 94.17%, respectively. The performance of the DNN model was not as accurate as other models, but the training speed was fast. The LSTM model was not as stable as the CNN and CNN-LSTM models. Moreover, with the same number of parameters, the training speed of the LSTM was much slower and it was difficult to achieve convergence. Additional parameter comparison experiments with other models, including epoch, learning rate, and dropout probability, were also conducted in the paper. Comparison results prove that the DNN model converged to optimal with fewer epochs and a higher learning rate. In contrast, the CNN model needed more epochs to learn. As for dropout probability, reducing the parameters by ~50% each time was appropriate.

scholarly journals Optimizing Convolution Neural Network on the TI C6678 multicore DSP

2018 ◽  
Vol 246 ◽  
pp. 03044 ◽  
Author(s):  
Guozhao Zeng ◽  
Xiao Hu ◽  
Yueyue Chen
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Image Processing ◽  
Deep Learning ◽  
Digital Signal Processor ◽  
High Performance ◽  
Digital Signal ◽  
Automatic Translation ◽  
Convolution Operation ◽  
Multicore Dsp

Convolutional Neural Networks (CNNs) have become the most advanced algorithms for deep learning. They are widely used in image processing, object detection and automatic translation. As the demand for CNNs continues to increase, the platforms on which they are deployed continue to expand. As an excellent low-power, high-performance, embedded solution, Digital Signal Processor (DSP) is used frequently in many key areas. This paper attempts to deploy the CNN to Texas Instruments (TI)’s TMS320C6678 multi-core DSP and optimize the main operations (convolution) to accommodate the DSP structure. The efficiency of the improved convolution operation has increased by tens of times.

scholarly journals Evaluation of Different Machine Learning Methods and Deep-Learning Convolutional Neural Networks for Landslide Detection

2019 ◽  
Vol 11 (2) ◽  
pp. 196 ◽  
Author(s):  
Omid Ghorbanzadeh ◽  
Thomas Blaschke ◽  
Khalil Gholamnia ◽  
Sansar Meena ◽  
Dirk Tiede ◽  
...  
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Deep Learning ◽  
Expert Knowledge ◽  
Accuracy Assessment ◽  
Window Size ◽  
Optical Data ◽  
Support Vector ◽  
Learning Methods ◽  
Machine Learning Methods

There is a growing demand for detailed and accurate landslide maps and inventories around the globe, but particularly in hazard-prone regions such as the Himalayas. Most standard mapping methods require expert knowledge, supervision and fieldwork. In this study, we use optical data from the Rapid Eye satellite and topographic factors to analyze the potential of machine learning methods, i.e., artificial neural network (ANN), support vector machines (SVM) and random forest (RF), and different deep-learning convolution neural networks (CNNs) for landslide detection. We use two training zones and one test zone to independently evaluate the performance of different methods in the highly landslide-prone Rasuwa district in Nepal. Twenty different maps are created using ANN, SVM and RF and different CNN instantiations and are compared against the results of extensive fieldwork through a mean intersection-over-union (mIOU) and other common metrics. This accuracy assessment yields the best result of 78.26% mIOU for a small window size CNN, which uses spectral information only. The additional information from a 5 m digital elevation model helps to discriminate between human settlements and landslides but does not improve the overall classification accuracy. CNNs do not automatically outperform ANN, SVM and RF, although this is sometimes claimed. Rather, the performance of CNNs strongly depends on their design, i.e., layer depth, input window sizes and training strategies. Here, we conclude that the CNN method is still in its infancy as most researchers will either use predefined parameters in solutions like Google TensorFlow or will apply different settings in a trial-and-error manner. Nevertheless, deep-learning can improve landslide mapping in the future if the effects of the different designs are better understood, enough training samples exist, and the effects of augmentation strategies to artificially increase the number of existing samples are better understood.

Sign in / Sign up

Export Citation Format

Share Document