scholarly journals Structural Analysis and Optimization of Convolutional Neural Networks with a Small Sample Size

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Rhett N. D’souza ◽  
Po-Yao Huang ◽  
Fang-Cheng Yeh
Keyword(s):  
Neural Networks ◽  
Structural Analysis ◽  
Sample Size ◽  
Convolutional Neural Networks ◽  
Small Sample Size ◽  
Small Sample

scholarly journals Rectified Wing Loss for Efficient and Robust Facial Landmark Localisation with Convolutional Neural Networks

2019 ◽  
Vol 128 (8-9) ◽  
pp. 2126-2145 ◽  
Author(s):  
Zhen-Hua Feng ◽  
Josef Kittler ◽  
Muhammad Awais ◽  
Xiao-Jun Wu
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Loss Function ◽  
Data Augmentation ◽  
Small Sample Size ◽  
Small Sample ◽  
Imbalance Problem ◽  
Facial Landmark ◽  
The Impact ◽  
Coarse To Fine

AbstractEfficient and robust facial landmark localisation is crucial for the deployment of real-time face analysis systems. This paper presents a new loss function, namely Rectified Wing (RWing) loss, for regression-based facial landmark localisation with Convolutional Neural Networks (CNNs). We first systemically analyse different loss functions, including L2, L1 and smooth L1. The analysis suggests that the training of a network should pay more attention to small-medium errors. Motivated by this finding, we design a piece-wise loss that amplifies the impact of the samples with small-medium errors. Besides, we rectify the loss function for very small errors to mitigate the impact of inaccuracy of manual annotation. The use of our RWing loss boosts the performance significantly for regression-based CNNs in facial landmarking, especially for lightweight network architectures. To address the problem of under-representation of samples with large pose variations, we propose a simple but effective boosting strategy, referred to as pose-based data balancing. In particular, we deal with the data imbalance problem by duplicating the minority training samples and perturbing them by injecting random image rotation, bounding box translation and other data augmentation strategies. Last, the proposed approach is extended to create a coarse-to-fine framework for robust and efficient landmark localisation. Moreover, the proposed coarse-to-fine framework is able to deal with the small sample size problem effectively. The experimental results obtained on several well-known benchmarking datasets demonstrate the merits of our RWing loss and prove the superiority of the proposed method over the state-of-the-art approaches.

Transfer learning with convolutional neural networks for small sample size problem in machinery fault diagnosis

2019 ◽  
Vol 233 (14) ◽  
pp. 5131-5143 ◽  
Author(s):  
Dengyu Xiao ◽  
Yixiang Huang ◽  
Chengjin Qin ◽  
Zhiyu Liu ◽  
Yanming Li ◽  
...  
Keyword(s):  
Neural Networks ◽  
Fault Diagnosis ◽  
Sample Size ◽  
Transfer Learning ◽  
Small Sample Size ◽  
Small Sample ◽  
Diagnostic Model ◽  
Source Data ◽  
Target Data ◽  
Machinery Fault Diagnosis

Data-driven machinery fault diagnosis has gained much attention from academic research and industry to guarantee the machinery reliability. Traditional fault diagnosis frameworks are commonly under a default assumption: the training and test samples share the similar distribution. However, it is nearly impossible in real industrial applications, where the operating condition always changes over time and the quantity of the same-distribution samples is often not sufficient to build a qualified diagnostic model. Therefore, transfer learning, which possesses the capacity to leverage the knowledge learnt from the massive source data to establish a diagnosis model for the similar but small target data, has shown potential value in machine fault diagnosis with small sample size. In this paper, we propose a novel fault diagnosis framework for the small amount of target data based on transfer learning, using a modified TrAdaBoost algorithm and convolutional neural networks. First, the massive source data with different distributions is added to the target data as the training data. Then, a convolutional neural network is selected as the base learner and the modified TrAdaBoost algorithm is employed for the weight update of each training sample to form a stronger diagnostic model. The whole proposition is experimentally demonstrated and discussed by carrying out the tests of six three-phase induction motors under different operating conditions and fault types. Results show that compared with other methods, the proposed framework can achieve the highest fault diagnostic accuracy with inadequate target data.

scholarly journals Small Data Challenge: Structural Analysis and Optimization of Convolutional Neural Networks with a Small Sample Size

2018 ◽  
Author(s):  
Rhett N. D’souza ◽  
Po-Yao Huang ◽  
Fang-Cheng Yeh
Keyword(s):  
Neural Networks ◽  
Big Data ◽  
Sample Size ◽  
Network Structure ◽  
Small Sample Size ◽  
Small Sample ◽  
Small Data ◽  
Optimal Network ◽  
Training Samples ◽  
Improved Accuracy

AbstractDeep neural networks have gained immense popularity in the Big Data problem; however, the availability of training samples can be relatively limited in certain application domains, particularly medical imaging, and consequently leading to overfitting problems. This “Small Data” challenge may need a mindset that is entirely different from the existing Big Data paradigm. Here, under the small data setting, we examined whether the network structure has a substantial influence on the performance and whether the optimal structure is predominantly determined by sample size or data nature. To this end, we listed all possible combinations of layers given an upper bound of the VC-dimension to study how structural hyperparameters affected the performance. Our results showed that structural optimization improved accuracy by 27.99%, 16.44%, and 13.11% over random selection for a sample size of 100, 500, and 1,000 in the MNIST dataset, respectively, suggesting that the importance of the network structure increases as the sample size becomes smaller. Furthermore, the optimal network structure was mostly determined by the data nature (photographic, calligraphic, or medical images), and less affected by the sample size, suggesting that the optimal network structure is data-driven, not sample size driven. After network structure optimization, the conventional convolutional neural network could achieve 91.13% in accuracy with only 500 samples, 93.66% in accuracy with only 1000 samples for the MNIST dataset and 94.10% in accuracy with only 3300 samples for the Mitosis (microscopic) dataset. These results indicate the primary importance of the network structure and the nature of the data in facing the Small Data challenge.

Application of Ultrasound Elastography for Assessing Intestinal Fibrosis in Inflammatory Bowel Disease: Fiction or Reality?

2020 ◽  
Vol 21 ◽  
Author(s):  
Roberto Gabbiadini ◽  
Eirini Zacharopoulou ◽  
Federica Furfaro ◽  
Vincenzo Craviotto ◽  
Alessandra Zilli ◽  
...  
Keyword(s):  
Inflammatory Bowel Disease ◽  
Sample Size ◽  
Bowel Disease ◽  
Small Sample Size ◽  
Shear Wave Elastography ◽  
Small Sample ◽  
Ultrasound Elastography ◽  
Invasive Technique ◽  
Intestinal Fibrosis ◽  
Inflammatory Bowel

Background: Intestinal fibrosis and subsequent strictures represent an important burden in inflammatory bowel disease (IBD). The detection and evaluation of the degree of fibrosis in stricturing Crohn’s disease (CD) is important to address the best therapeutic strategy (medical anti-inflammatory therapy, endoscopic dilation, surgery). Ultrasound elastography (USE) is a non-invasive technique that has been proposed in the field of IBD for evaluating intestinal stiffness as a biomarker of intestinal fibrosis. Objective: The aim of this review is to discuss the ability and current role of ultrasound elastography in the assessment of intestinal fibrosis. Results and Conclusion: Data on USE in IBD are provided by pilot and proof-of-concept studies with small sample size. The first type of USE investigated was strain elastography, while shear wave elastography has been introduced lately. Despite the heterogeneity of the methods of the studies, USE has been proven to be able to assess intestinal fibrosis in patients with stricturing CD. However, before introducing this technique in current practice, further studies with larger sample size and homogeneous parameters, testing reproducibility, and identification of validated cut-off values are needed.

scholarly journals Confirming known planetary trends using a photometrically selected Kepler sample

2020 ◽  
Author(s):  
Jonah T Hansen ◽  
Luca Casagrande ◽  
Michael J Ireland ◽  
Jane Lin
Keyword(s):  
Sample Size ◽  
Robust Statistics ◽  
Narrow Band ◽  
Small Sample Size ◽  
Small Sample ◽  
Fitting Algorithm ◽  
Statistical Studies ◽  
The Rich ◽  
Host Stars ◽  
High Metallicity

Abstract Statistical studies of exoplanets and the properties of their host stars have been critical to informing models of planet formation. Numerous trends have arisen in particular from the rich Kepler dataset, including that exoplanets are more likely to be found around stars with a high metallicity and the presence of a “gap” in the distribution of planetary radii at 1.9 R⊕. Here we present a new analysis on the Kepler field, using the APOGEE spectroscopic survey to build a metallicity calibration based on Gaia, 2MASS and Strömgren photometry. This calibration, along with masses and radii derived from a Bayesian isochrone fitting algorithm, is used to test a number of these trends with unbiased, photometrically derived parameters, albeit with a smaller sample size in comparison to recent studies. We recover that planets are more frequently found around higher metallicity stars; over the entire sample, planetary frequencies are 0.88 ± 0.12 percent for [Fe/H] < 0 and 1.37 ± 0.16 percent for [Fe/H] ≥ 0 but at two sigma we find that the size of exoplanets influences the strength of this trend. We also recover the planet radius gap, along with a slight positive correlation with stellar mass. We conclude that this method shows promise to derive robust statistics of exoplanets. We also remark that spectrophotometry from Gaia DR3 will have an effective resolution similar to narrow band filters and allow to overcome the small sample size inherent in this study.

Sign in / Sign up

Export Citation Format

Share Document