Boosting neural networks in real world applications: An empirical study

1997 ◽  
pp. 321-329
Author(s):  
Hongxing He ◽  
Zhexue Huang
Keyword(s):  
Neural Networks ◽  
Empirical Study ◽  
Real World ◽  
Real World Applications

scholarly journals INDUSTRIAL APPLICATIONS OF ARTIFICIAL NEURAL NETWORKS

2014 ◽  
pp. 8-20
Author(s):  
Kurosh Madani
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Real World ◽  
Industrial Applications ◽  
Large Field ◽  
Nonlinear Functions ◽  
The Past ◽  
Real World Applications ◽  
Ann Models ◽  
Artificial Neural

In a large number of real world dilemmas and related applications the modeling of complex behavior is the central point. Over the past decades, new approaches based on Artificial Neural Networks (ANN) have been proposed to solve problems related to optimization, modeling, decision making, classification, data mining or nonlinear functions (behavior) approximation. Inspired from biological nervous systems and brain structure, Artificial Neural Networks could be seen as information processing systems, which allow elaboration of many original techniques covering a large field of applications. Among their most appealing properties, one can quote their learning and generalization capabilities. The main goal of this paper is to present, through some of main ANN models and based techniques, their real application capability in real world industrial dilemmas. Several examples through industrial and real world applications have been presented and discussed.

Chapter 15. Human-Centered Concept Explanations for Neural Networks

2021 ◽  
Author(s):  
Chih-Kuan Yeh ◽  
Been Kim ◽  
Pradeep Ravikumar
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Case Studies ◽  
Real World ◽  
Deep Neural Networks ◽  
Learning Models ◽  
Real World Applications ◽  
The Right ◽  
Concept Activation ◽  
Machine Learning Models

Understanding complex machine learning models such as deep neural networks with explanations is crucial in various applications. Many explanations stem from the model perspective, and may not necessarily effectively communicate why the model is making its predictions at the right level of abstraction. For example, providing importance weights to individual pixels in an image can only express which parts of that particular image is important to the model, but humans may prefer an explanation which explains the prediction by concept-based thinking. In this work, we review the emerging area of concept based explanations. We start by introducing concept explanations including the class of Concept Activation Vectors (CAV) which characterize concepts using vectors in appropriate spaces of neural activations, and discuss different properties of useful concepts, and approaches to measure the usefulness of concept vectors. We then discuss approaches to automatically extract concepts, and approaches to address some of their caveats. Finally, we discuss some case studies that showcase the utility of such concept-based explanations in synthetic settings and real world applications.

scholarly journals Transfer Learning and Deep Domain Adaptation

2020 ◽  
Author(s):  
Wen Xu ◽  
Jing He ◽  
Yanfeng Shu
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Transfer Learning ◽  
Real World ◽  
Deep Neural Networks ◽  
Domain Adaptation ◽  
Fine Tuning ◽  
Real World Applications ◽  
Comprehensive Survey ◽  
Sample Reconstruction

Transfer learning is an emerging technique in machine learning, by which we can solve a new task with the knowledge obtained from an old task in order to address the lack of labeled data. In particular deep domain adaptation (a branch of transfer learning) gets the most attention in recently published articles. The intuition behind this is that deep neural networks usually have a large capacity to learn representation from one dataset and part of the information can be further used for a new task. In this research, we firstly present the complete scenarios of transfer learning according to the domains and tasks. Secondly, we conduct a comprehensive survey related to deep domain adaptation and categorize the recent advances into three types based on implementing approaches: fine-tuning networks, adversarial domain adaptation, and sample-reconstruction approaches. Thirdly, we discuss the details of these methods and introduce some typical real-world applications. Finally, we conclude our work and explore some potential issues to be further addressed.

An Empirical Study of Boosting Methods on Severely Imbalanced Data

2014 ◽  
Vol 513-517 ◽  
pp. 2510-2513 ◽  
Author(s):  
Xu Ying Liu
Keyword(s):  
Empirical Study ◽  
Real World ◽  
Class Imbalance ◽  
Imbalanced Data ◽  
Real World Applications ◽  
Under Sampling ◽  
The Difference ◽  
Imbalance Learning ◽  
Class Imbalance Learning ◽  
F Measure

Nowadays there are large volumes of data in real-world applications, which poses great challenge to class-imbalance learning: the large amount of the majority class examples and severe class-imbalance. Previous studies on class-imbalance learning mainly focused on relatively small or moderate class-imbalance. In this paper we conduct an empirical study to explore the difference between learning with small or moderate class-imbalance and learning with severe class-imbalance. The experimental results show that: (1) Traditional methods cannot handle severe class-imbalance effectively. (2) AUC, G-mean and F-measure can be very inconsistent for severe class-imbalance, which seldom appears when class-imbalance is moderate. And G-mean is not appropriate for severe class-imbalance learning because it is not sensitive to the change of imbalance ratio. (3) When AUC and G-mean are evaluation metrics, EasyEnsemble is the best method, followed by BalanceCascade and under-sampling. (4) A little under-full balance is better for under-sampling to handle severe class-imbalance. And it is important to handle false positives when design methods for severe class-imbalance.

PRACTICAL PROBLEMS USING NEURAL NETWORKS

1992 ◽  
Vol 03 (supp01) ◽  
pp. 25-30
Author(s):  
Françoise Fogelman Soulie
Keyword(s):  
Neural Networks ◽  
Real World ◽  
Real World Applications

Neural Networks are very efficient for real world applications. However, practical problems often arise which can hinder performances. We discuss here some of these problems: under-representation of classes, rejection of outliers and ambiguous patterns, and illustrate the issues raised through various applications.

scholarly journals Understanding Generalization in Neural Networks for Robustness against Adversarial Vulnerabilities

2020 ◽  
Vol 34 (10) ◽  
pp. 13714-13715
Author(s):  
Subhajit Chaudhury
Keyword(s):  
Neural Networks ◽  
Computer Vision ◽  
Real World ◽  
Speech Processing ◽  
State Of The Art ◽  
Feature Learning ◽  
Equivalent Problem ◽  
Tremendous Progress ◽  
Real World Applications ◽  
Out Of Sample

Neural networks have contributed to tremendous progress in the domains of computer vision, speech processing, and other real-world applications. However, recent studies have shown that these state-of-the-art models can be easily compromised by adding small imperceptible perturbations. My thesis summary frames the problem of adversarial robustness as an equivalent problem of learning suitable features that leads to good generalization in neural networks. This is motivated from learning in humans which is not trivially fooled by such perturbations due to robust feature learning which shows good out-of-sample generalization.

scholarly journals EHANet: An Effective Hierarchical Aggregation Network for Face Parsing

2020 ◽  
Vol 10 (9) ◽  
pp. 3135 ◽  
Author(s):  
Ling Luo ◽  
Dingyu Xue ◽  
Xinglong Feng
Keyword(s):  
Neural Networks ◽  
Real World ◽  
State Of The Art ◽  
Contextual Information ◽  
Semantic Gap ◽  
Deep Convolutional Neural Networks ◽  
Multi Scale ◽  
Hierarchical Aggregation ◽  
Real World Applications ◽  
Weighted Boundary

In recent years, benefiting from deep convolutional neural networks (DCNNs), face parsing has developed rapidly. However, it still has the following problems: (1) Existing state-of-the-art frameworks usually do not satisfy real-time while pursuing performance; (2) similar appearances cause incorrect pixel label assignments, especially in the boundary; (3) to promote multi-scale prediction, deep features and shallow features are used for fusion without considering the semantic gap between them. To overcome these drawbacks, we propose an effective and efficient hierarchical aggregation network called EHANet for fast and accurate face parsing. More specifically, we first propose a stage contextual attention mechanism (SCAM), which uses higher-level contextual information to re-encode the channel according to its importance. Secondly, a semantic gap compensation block (SGCB) is presented to ensure the effective aggregation of hierarchical information. Thirdly, the advantages of weighted boundary-aware loss effectively make up for the ambiguity of boundary semantics. Without any bells and whistles, combined with a lightweight backbone, we achieve outstanding results on both CelebAMask-HQ (78.19% mIoU) and Helen datasets (90.7% F1-score). Furthermore, our model can achieve 55 FPS on a single GTX 1080Ti card with 640 × 640 input and further reach over 300 FPS with a resolution of 256 × 256, which is suitable for real-world applications.

scholarly journals Feature Statistics Guided Efficient Filter Pruning

2020 ◽  
Author(s):  
Hang Li ◽  
Chen Ma ◽  
Wei Xu ◽  
Xue Liu
Keyword(s):  
Neural Networks ◽  
Real World ◽  
Feature Maps ◽  
L1 Norm ◽  
Feature Map ◽  
Single Feature ◽  
Real World Applications ◽  
Information Diversity ◽  
Reliable Performance ◽  
Pruning Methods

Building compact convolutional neural networks (CNNs) with reliable performance is a critical but challenging task, especially when deploying them in real-world applications. As a common approach to reduce the size of CNNs, pruning methods delete part of the CNN filters according to some metrics such as l1-norm. However, previous methods hardly leverage the information variance in a single feature map and the similarity characteristics among feature maps. In this paper, we propose a novel filter pruning method, which incorporates two kinds of feature map selections: diversity-aware selection (DFS) and similarity-aware selection (SFS). DFS aims to discover features with low information diversity while SFS removes features that have high similarities with others. We conduct extensive empirical experiments with various CNN architectures on publicly available datasets. The experimental results demonstrate that our model obtains up to 91.6% parameter decrease and 83.7% FLOPs reduction with almost no accuracy loss.

Sign in / Sign up

Export Citation Format

Share Document