scholarly journals AMBIENT: Accelerated Convolutional Neural Network Architecture Search for Regulatory Genomics

2021 ◽  
Author(s):  
Zijun Zhang ◽  
Evan M. Cofer ◽  
Olga G. Troyanskaya
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Network Architecture ◽  
Environmental Issue ◽  
Biological Sequences ◽  
Neural Network Architecture ◽  
Computing Power ◽  
Neural Architecture

Convolutional neural networks (CNN) have become a standard approach for modeling genomic sequences. CNNs can be effectively built by Neural Architecture Search (NAS) by trading computing power for accurate neural architectures. Yet, the consumption of immense computing power is a major practical, financial, and environmental issue for deep learning. Here, we present a novel NAS framework, AMBIENT, that generates highly accurate CNN architectures for biological sequences of diverse functions, while substantially reducing the computing cost of conventional NAS.

scholarly journals Classification of Skin Disease Using Deep Learning Neural Networks with MobileNet V2 and LSTM

Sensors ◽  
2021 ◽  
Vol 21 (8) ◽  
pp. 2852
Author(s):  
Parvathaneni Naga Srinivasu ◽  
Jalluri Gnana SivaSai ◽  
Muhammad Fazal Ijaz ◽  
Akash Kumar Bhoi ◽  
Wonjoon Kim ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Skin Disease ◽  
Network Architecture ◽  
Large Scale ◽  
Short Term Memory ◽  
Convolutional Networks ◽  
Occurrence Matrix

Deep learning models are efficient in learning the features that assist in understanding complex patterns precisely. This study proposed a computerized process of classifying skin disease through deep learning based MobileNet V2 and Long Short Term Memory (LSTM). The MobileNet V2 model proved to be efficient with a better accuracy that can work on lightweight computational devices. The proposed model is efficient in maintaining stateful information for precise predictions. A grey-level co-occurrence matrix is used for assessing the progress of diseased growth. The performance has been compared against other state-of-the-art models such as Fine-Tuned Neural Networks (FTNN), Convolutional Neural Network (CNN), Very Deep Convolutional Networks for Large-Scale Image Recognition developed by Visual Geometry Group (VGG), and convolutional neural network architecture that expanded with few changes. The HAM10000 dataset is used and the proposed method has outperformed other methods with more than 85% accuracy. Its robustness in recognizing the affected region much faster with almost 2× lesser computations than the conventional MobileNet model results in minimal computational efforts. Furthermore, a mobile application is designed for instant and proper action. It helps the patient and dermatologists identify the type of disease from the affected region’s image at the initial stage of the skin disease. These findings suggest that the proposed system can help general practitioners efficiently and effectively diagnose skin conditions, thereby reducing further complications and morbidity.

scholarly journals Designing deep neural networks for continual learning in an open world

2021 ◽  
Author(s):  
◽  
Martin Mundt
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Deep Learning ◽  
Network Architecture ◽  
Neural Network Training ◽  
Neural Network Architecture ◽  
Neural Architecture ◽  
Network Training ◽  
Classification Tasks ◽  
Continual Learning

Deep learning with neural networks seems to have largely replaced traditional design of computer vision systems. Automated methods to learn a plethora of parameters are now used in favor of previously practiced selection of explicit mathematical operators for a specific task. The entailed promise is that practitioners no longer need to take care of every individual step, but rather focus on gathering big amounts of data for neural network training. As a consequence, both a shift in mindset towards a focus on big datasets, as well as a wave of conceivable applications based exclusively on deep learning can be observed. This PhD dissertation aims to uncover some of the only implicitly mentioned or overlooked deep learning aspects, highlight unmentioned assumptions, and finally introduce methods to address respective immediate weaknesses. In the author’s humble opinion, these prevalent shortcomings can be tied to the fact that the involved steps in the machine learning workflow are frequently decoupled. Success is predominantly measured based on accuracy measures designed for evaluation with static benchmark test sets. Individual machine learning workflow components are assessed in isolation with respect to available data, choice of neural network architecture, and a particular learning algorithm, rather than viewing the machine learning system as a whole in context of a particular application. Correspondingly, in this dissertation, three key challenges have been identified: 1. Choice and flexibility of a neural network architecture. 2. Identification and rejection of unseen unknown data to avoid false predictions. 3. Continual learning without forgetting of already learned information. These latter challenges have already been crucial topics in older literature, alas, seem to require a renaissance in modern deep learning literature. Initially, it may appear that they pose independent research questions, however, the thesis posits that the aspects are intertwined and require a joint perspective in machine learning based systems. In summary, the essential question is thus how to pick a suitable neural network architecture for a specific task, how to recognize which data inputs belong to this context, which ones originate from potential other tasks, and ultimately how to continuously include such identified novel data in neural network training over time without overwriting existing knowledge. Thus, the central emphasis of this dissertation is to build on top of existing deep learning strengths, yet also acknowledge mentioned weaknesses, in an effort to establish a deeper understanding of interdependencies and synergies towards the development of unified solution mechanisms. For this purpose, the main portion of the thesis is in cumulative form. The respective publications can be grouped according to the three challenges outlined above. Correspondingly, chapter 1 is focused on choice and extendability of neural network architectures, analyzed in context of popular image classification tasks. An algorithm to automatically determine neural network layer width is introduced and is first contrasted with static architectures found in the literature. The importance of neural architecture design is then further showcased on a real-world application of defect detection in concrete bridges. Chapter 2 is comprised of the complementary ensuing questions of how to identify unknown concepts and subsequently incorporate them into continual learning. A joint central mechanism to distinguish unseen concepts from what is known in classification tasks, while enabling consecutive training without forgetting or revisiting older classes, is proposed. Once more, the role of the chosen neural network architecture is quantitatively reassessed. Finally, chapter 3 culminates in an overarching view, where developed parts are connected. Here, an extensive survey further serves the purpose to embed the gained insights in the broader literature landscape and emphasizes the importance of a common frame of thought. The ultimately presented approach thus reflects the overall thesis’ contribution to advance neural network based machine learning towards a unified solution that ties together choice of neural architecture with the ability to learn continually and the capability to automatically separate known from unknown data.

Application of convolutional neural networks for monitoring of marine objects

2020 ◽  
pp. 53-61
Author(s):  
Н.А. Полковникова ◽  
Е.В. Тузинкевич ◽  
А.Н. Попов
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Computer Vision ◽  
Convolutional Neural Network ◽  
Convolutional Neural Networks ◽  
Network Architecture ◽  
Research Process ◽  
Probability Of Detection ◽  
Neural Network Architecture ◽  
Deep Convolutional Neural Networks

В статье рассмотрены технологии компьютерного зрения на основе глубоких свёрточных нейронных сетей. Применение нейронных сетей особенно эффективно для решения трудно формализуемых задач. Разработана архитектура свёрточной нейронной сети применительно к задаче распознавания и классификации морских объектов на изображениях. В ходе исследования выполнен ретроспективный анализ технологий компьютерного зрения и выявлен ряд проблем, связанных с применением нейронных сетей: «исчезающий» градиент, переобучение и вычислительная сложность. При разработке архитектуры нейросети предложено использовать функцию активации RELU, обучение некоторых случайно выбранных нейронов и нормализацию с целью упрощения архитектуры нейросети. Сравнение используемых в нейросети функций активации ReLU, LeakyReLU, Exponential ReLU и SOFTMAX выполнено в среде Matlab R2020a. На основе свёрточной нейронной сети разработана программа на языке программирования Visual C# в среде MS Visual Studio для распознавания морских объектов. Программапредназначена для автоматизированной идентификации морских объектов, производит детектирование (нахождение объектов на изображении) и распознавание объектов с высокой вероятностью обнаружения. The article considers computer vision technologies based on deep convolutional neural networks. Application of neural networks is particularly effective for solving difficult formalized problems. As a result convolutional neural network architecture to the problem of recognition and classification of marine objects on images is implemented. In the research process a retrospective analysis of computer vision technologies was performed and a number of problems associated with the use of neural networks were identified: vanishing gradient, overfitting and computational complexity. To solve these problems in neural network architecture development, it was proposed to use RELU activation function, training some randomly selected neurons and normalization for simplification of neural network architecture. Comparison of ReLU, LeakyReLU, Exponential ReLU, and SOFTMAX activation functions used in the neural network implemented in Matlab R2020a.The computer program based on convolutional neural network for marine objects recognition implemented in Visual C# programming language in MS Visual Studio integrated development environment. The program is designed for automated identification of marine objects, produces detection (i.e., presence of objects on image), and objects recognition with high probability of detection.

scholarly journals ThriftyNets: Convolutional Neural Networks with Tiny Parameter Budget

IoT ◽  
2021 ◽  
Vol 2 (2) ◽  
pp. 222-235
Author(s):  
Guillaume Coiffier ◽  
Ghouthi Boukli Hacene ◽  
Vincent Gripon
Keyword(s):  
Neural Network ◽  
Machine Learning ◽  
Neural Networks ◽  
Convolutional Neural Network ◽  
Spatial Resolution ◽  
Network Architecture ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Feature Maps ◽  
Neural Network Architecture

Deep Neural Networks are state-of-the-art in a large number of challenges in machine learning. However, to reach the best performance they require a huge pool of parameters. Indeed, typical deep convolutional architectures present an increasing number of feature maps as we go deeper in the network, whereas spatial resolution of inputs is decreased through downsampling operations. This means that most of the parameters lay in the final layers, while a large portion of the computations are performed by a small fraction of the total parameters in the first layers. In an effort to use every parameter of a network at its maximum, we propose a new convolutional neural network architecture, called ThriftyNet. In ThriftyNet, only one convolutional layer is defined and used recursively, leading to a maximal parameter factorization. In complement, normalization, non-linearities, downsamplings and shortcut ensure sufficient expressivity of the model. ThriftyNet achieves competitive performance on a tiny parameters budget, exceeding 91% accuracy on CIFAR-10 with less than 40 k parameters in total, 74.3% on CIFAR-100 with less than 600 k parameters, and 67.1% On ImageNet ILSVRC 2012 with no more than 4.15 M parameters. However, the proposed method typically requires more computations than existing counterparts.

scholarly journals A Deep Convolutional Neural Network Architecture for Cancer Diagnosis using Histopathological Images

2021 ◽  
Vol 10 (12) ◽  
pp. 7-12
Author(s):  
Karthika Gidijala ◽  
◽  
Mansa Devi Pappu ◽  
Manasa Vavilapalli ◽  
Mahesh Kothuru ◽  
...  
Keyword(s):  
Breast Cancer ◽  
Neural Network ◽  
Neural Networks ◽  
Decision Making ◽  
Deep Learning ◽  
Cancer Diagnosis ◽  
Network Architecture ◽  
Neural Network Architecture ◽  
Proposed Model ◽  
Histopathological Images

Many different models of Convolution Neural Networks exist in the Deep Learning studies. The application and prudence of the algorithms is known only when they are implemented with strong datasets. The histopathological images of breast cancer are considered as to have much number of haphazard structures and textures. Dealing with such images is a challenging issue in deep learning. Working on wet labs and in coherence to the results many research have blogged with novel annotations in the research. In this paper, we are presenting a model that can work efficiently on the raw images with different resolutions and alleviating with the problems of the presence of the structures and textures. The proposed model achieves considerably good results useful for decision making in cancer diagnosis.

scholarly journals Neural Layer Bypassing Network

2021 ◽  
Author(s):  
Amogh Palasamudram
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Image Classification ◽  
Network Architecture ◽  
Classification Models ◽  
Learning Models ◽  
Neural Network Architecture ◽  
Overall Performance ◽  
The Impact

<p>This research aims to introduce and evaluate a new neural network architecture to improve the speed and effectiveness of forward propagation in neural networks: the Neural Layer Bypassing Network (NLBN). The theory and workings of this architecture have been explained in this research paper, along with comparisons to other methods of increasing the efficacy of deep learning models. This research also includes code examples with 3 image classification models trained on different datasets and analyses the impact of the NLBN architecture on forward propagation. It was found that this architecture increases the speed of forward propagation and tends to slightly decrease the accuracy of the model. However, it takes longer to train and takes more memory. All in all, this architecture is a potential foundation for using deep learning to teach deep learning models to be more efficient. This includes skipping and re-propagating through layers to improve the overall performance of a model.</p><div><br></div>

scholarly journals Sistem Deteksi Infeksi COVID-19 Pada Hasil X-Ray Rontgen menggunakan Algoritma Convolutional Neural Network (CNN)

2021 ◽  
Vol 4 (2) ◽  
pp. 217-227
Author(s):  
Muhammad Saiful ◽  
◽  
Lalu Muhammad Samsu ◽  
Fathurrahman Fathurrahman ◽  
◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Computer Vision ◽  
Artificial Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Network Architecture ◽  
Infection Detection ◽  
Learning Techniques ◽  
Artificial Neural

The development of the world's technology is growing rapidly, especially in the field of health in the form of detection tools of various objects, including disease objects. The technology in point is part of artificial intelligence that is able to recognize a set of imagery and classify automatically with deep learning techniques. One of the deep learning networks widely used is convolutional neural network with computer vision technology. One of the problems with computer vision that is still developing is object detection as a useful technology to recognize objects in the image as if humans knew the object of the image. In this case, a computer machine is trained in learning using artificial neural networks. One of the sub types of artificial neural networks that are able to handle computer vision problems is by using deep learning techniques with convolutional neural network algorithms. The purpose of this research is to find out how to design the system, the network architecture used for COVID-19 infection detection. The system cannot perform detection of other objects. The results of COVID-19 infection detection with convolutional neural network algorithm show unlimited accuracy value that ranges from 60-99%.

Malicious Code Invariance Based On Deep Learning

2021 ◽  
Vol 10 (3) ◽  
pp. 1-7
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Network Architecture ◽  
Malicious Code ◽  
Neural Network Architecture ◽  
Security Issues ◽  
The Neural Network ◽  
Code Identification ◽  
Malicious Code Detection

The malicious code detection is critical task for in the field of security. The malicious code detection can be possibly by using convolutional neural network (CNN).Themalicious code can be categorized in to different families. The malicious code identification helps to identify the affected malware on the system. Malicious code theft data from our system and it yields high security issues in real time. The neural network architecture classifies the malicious code based on the collected dataset. The dataset contains different families of malicious code. The malicious code detection can be done with the help of model created from CNN architecture

Sign in / Sign up

Export Citation Format

Share Document