A New Intra Fine-Tuning Method Between Histopathological Datasets in Deep Learning

2020 ◽  
Vol 11 (2) ◽  
pp. 16-40
Author(s):  
Nassima Dif ◽  
Zakaria Elberrichi
Keyword(s):  
Network Architecture ◽  
Source Model ◽  
Fine Tuning ◽  
Neural Network Architecture ◽  
High Quality ◽  
Tuning Method ◽  
Histopathological Images ◽  
The Comparative Study ◽  
First Time ◽  
Target Sets

This article presents a new fine-tuning framework for histopathological images analysis. Despite the most common solutions where the ImageNet models are reused for image classification, this research sets out to perform an intra-domain fine tuning between the trained models on the histopathological images. The purpose is to take advantage of the hypothesis on the efficiency of transfer learning between non-distant datasets and to examine for the first time these suggestions on the histopathological images. The Inception-v3 convolutional neural network architecture, six histopathological source datasets, and four target sets as base modules were used in this article. The obtained results reveal the importance of the pre-trained histopathological models compared to the ImageNet model. In particular, the ICIAR 2018-A presented a high-quality source model for the various target tasks due to its capacity in generalization. Finally, the comparative study with the other literature results shows that the proposed method achieved the best results on both CRC (95.28%) and KIMIA-PATH (98.18%) datasets.

scholarly journals Backbone Cannot Be Trained at Once: Rolling Back to Pre-Trained Network for Person Re-Identification

2019 ◽  
Vol 33 ◽  
pp. 8859-8867 ◽  
Author(s):  
Youngmin Ro ◽  
Jongwon Choi ◽  
Dae Ung Jo ◽  
Byeongho Heo ◽  
Jongin Lim ◽  
...  
Keyword(s):  
Network Architecture ◽  
State Of The Art ◽  
Fine Tuning ◽  
Neural Network Architecture ◽  
Large Dataset ◽  
Low Level ◽  
Tuning Method ◽  
Improved Performance ◽  
High Level ◽  
Tuning Strategy

In person re-identification (ReID) task, because of its shortage of trainable dataset, it is common to utilize fine-tuning method using a classification network pre-trained on a large dataset. However, it is relatively difficult to sufficiently finetune the low-level layers of the network due to the gradient vanishing problem. In this work, we propose a novel fine-tuning strategy that allows low-level layers to be sufficiently trained by rolling back the weights of high-level layers to their initial pre-trained weights. Our strategy alleviates the problem of gradient vanishing in low-level layers and robustly trains the low-level layers to fit the ReID dataset, thereby increasing the performance of ReID tasks. The improved performance of the proposed strategy is validated via several experiments. Furthermore, without any addons such as pose estimation or segmentation, our strategy exhibits state-of-the-art performance using only vanilla deep convolutional neural network architecture.

scholarly journals Kinematic Control of Manipulator with Remote Center of Motion Constraints Synthesised by a Simplified Recurrent Neural Network

2021 ◽  
Author(s):  
Zhan Li ◽  
Shuai Li
Keyword(s):  
Neural Network ◽  
Recurrent Neural Network ◽  
Network Architecture ◽  
Redundancy Resolution ◽  
Neural Network Architecture ◽  
Kinematic Control ◽  
Physical Constraints ◽  
Motion Constraints ◽  
Joint Limits ◽  
First Time

AbstractRedundancy manipulators need favorable redundancy resolution to obtain suitable control actions to guarantee accurate kinematic control. Among numerous kinematic control applications, some specific tasks such as minimally invasive manipulation/surgery require the distal link of a manipulator to translate along such fixed point. Such a point is known as remote center of motion (RCM) to constrain motion planning and kinematic control of manipulators. Recurrent neural network (RNN) which possesses parallel processing ability, is a powerful alternative and has achieved success in conventional redundancy resolution and kinematic control with physical constraints of joint limits. However, up to now, there still is few related works on the RNNs for redundancy resolution and kinematic control of manipulators with RCM constraints considered yet. In this paper, for the first time, an RNN-based approach with a simplified neural network architecture is proposed to solve the redundancy resolution issue with RCM constraints, with a new and general dynamic optimization formulation containing the RCM constraints investigated. Theoretical results analyze and convergence properties of the proposed simplified RNN for redundancy resolution of manipulators with RCM constraints. Simulation results further demonstrate the efficiency of the proposed method in end-effector path tracking control under RCM constraints based on a redundant manipulator.

scholarly journals A Deep Convolutional Neural Network Architecture for Cancer Diagnosis using Histopathological Images

2021 ◽  
Vol 10 (12) ◽  
pp. 7-12
Author(s):  
Karthika Gidijala ◽  
◽  
Mansa Devi Pappu ◽  
Manasa Vavilapalli ◽  
Mahesh Kothuru ◽  
...  
Keyword(s):  
Breast Cancer ◽  
Neural Network ◽  
Neural Networks ◽  
Decision Making ◽  
Deep Learning ◽  
Cancer Diagnosis ◽  
Network Architecture ◽  
Neural Network Architecture ◽  
Proposed Model ◽  
Histopathological Images

Many different models of Convolution Neural Networks exist in the Deep Learning studies. The application and prudence of the algorithms is known only when they are implemented with strong datasets. The histopathological images of breast cancer are considered as to have much number of haphazard structures and textures. Dealing with such images is a challenging issue in deep learning. Working on wet labs and in coherence to the results many research have blogged with novel annotations in the research. In this paper, we are presenting a model that can work efficiently on the raw images with different resolutions and alleviating with the problems of the presence of the structures and textures. The proposed model achieves considerably good results useful for decision making in cancer diagnosis.

Neural Networks Applications to Combustion Process Simulation

2021 ◽  
pp. 15-29
Author(s):  
Б. В. Крыжановский ◽  
Н. Н. Смирнов ◽  
В. Ф. Никитин ◽  
Я. М. Карандашев ◽  
М. Ю. Мальсагов ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Process Simulation ◽  
Network Architecture ◽  
Combustion Process ◽  
Fine Tuning ◽  
Neural Network Architecture ◽  
The Neural Network ◽  
Process Simulations

Моделирование горения является ключевым аспектом полномасштабного трехмерного моделирования современных и перспективных двигателей для авиационно-космических силовых установок. В данной работе изучается возможность решения задач химической кинетики с использованием искусственных нейронных сетей. С помощью классических численных методов были построены наборы обучающих данных. Выбирая среди различных архитектур многослойных нейронных сетей и настраивая их параметры, мы разработали достаточно простую модель, способную решить эту проблему. Полученная нейронная сеть работает в рекурсивном режиме и может предсказывать поведение химической многовидовой динамической системы за много шагов. Combustion process simulations are the key aspect enabling full-scale 3D simulations of advanced aerospace engines. This work studies solving chemical kinetics problems with artificial neural networks. The training datasets were generated by classical numerical methods. Choosing a multi-layer neural network architecture and fine-tuning its parameters, we developed a simple model that can solve the problem. The neural network obtained works is recursive, and by running many iterations it can predict the behavior of a chemical multimodal dynamic system.  

scholarly journals Optimizing Small BERTs Trained for German NER

Information ◽  
2021 ◽  
Vol 12 (11) ◽  
pp. 443
Author(s):  
Jochen Zöllner ◽  
Konrad Sperfeld ◽  
Christoph Wick ◽  
Roger Labahn
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Fine Tuning ◽  
Language Models ◽  
Memory Usage ◽  
Memory Consumption ◽  
Neural Network Architecture ◽  
Training Techniques

Currently, the most widespread neural network architecture for training language models is the so-called BERT, which led to improvements in various NLP tasks. In general, the larger the number of parameters in a BERT model, the better the results obtained in these NLP tasks. Unfortunately, the memory consumption and the training duration drastically increases with the size of these models. In this article, we investigate various training techniques of smaller BERT models: We combine different methods from other BERT variants, such as ALBERT, RoBERTa, and relative positional encoding. In addition, we propose two new fine-tuning modifications leading to better performance: CSE tagging and a modified form of LCRF. Furthermore, we introduce WWA, which reduces BERT memory usage and leads to a small increase in performance compared to classical Multi-Head-Attention. We evaluate these techniques on five public German NER tasks, of which two are introduced by this article.

SCORING MODELING BASED ON NEURAL NETWORKS FOR DETERMINING A BANK BORROWER'S RATING

2020 ◽  
Vol 2020 (10) ◽  
pp. 54-62
Author(s):  
Oleksii VASYLIEV ◽  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Network Architecture ◽  
Statistical Data ◽  
Activation Function ◽  
Decision Making Process ◽  
Neural Network Architecture ◽  
Acceptable Accuracy ◽  
The Neural Network ◽  
Sigmoid Activation Function

The problem of applying neural networks to calculate ratings used in banking in the decision-making process on granting or not granting loans to borrowers is considered. The task is to determine the rating function of the borrower based on a set of statistical data on the effectiveness of loans provided by the bank. When constructing a regression model to calculate the rating function, it is necessary to know its general form. If so, the task is to calculate the parameters that are included in the expression for the rating function. In contrast to this approach, in the case of using neural networks, there is no need to specify the general form for the rating function. Instead, certain neural network architecture is chosen and parameters are calculated for it on the basis of statistical data. Importantly, the same neural network architecture can be used to process different sets of statistical data. The disadvantages of using neural networks include the need to calculate a large number of parameters. There is also no universal algorithm that would determine the optimal neural network architecture. As an example of the use of neural networks to determine the borrower's rating, a model system is considered, in which the borrower's rating is determined by a known non-analytical rating function. A neural network with two inner layers, which contain, respectively, three and two neurons and have a sigmoid activation function, is used for modeling. It is shown that the use of the neural network allows restoring the borrower's rating function with quite acceptable accuracy.

Disability as a social phenomenon

2020 ◽  
pp. 22-38
Author(s):  
Natalia Guseva ◽  
Vitaliy Berdutin
Keyword(s):  
Social Protection ◽  
Russian Society ◽  
Stage Model ◽  
High Quality ◽  
Rehabilitation Programs ◽  
Current State ◽  
The Social ◽  
Consistent Solution ◽  
First Time

At present, the problem of establishing disability is a point at issue in Russia. Despite the fact that medical criteria for disability are being developed very actively, high-quality methods for assessing social hallmarks are still lacking. Since disability is a phenomenon inherent in any society, each state forms a social and economic policy for people with disabilities in accordance with its level of development, priorities and opportunities. We have proposed a three-stage model, which includes a system for the consistent solution of the main tasks aimed at studying the causes and consequences of the problems encountered today in the social protection of citizens with health problems. The article shows why the existing approaches to the determination of disability and rehabilitation programs do not correspond to the current state of Russian society and why a decrease in the rate of persons recognized as disabled for the first time does not indicate an improvement in the health of the population. The authors proposed a number of measures with a view to correcting the situation according to the results of the study.

Evolving Neural Network Architecture

1993 ◽  
Author(s):  
John R. McDonnell ◽  
Don Waagen
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Neural Network Architecture
Sign in / Sign up

Export Citation Format

Share Document