scholarly journals Small Data Challenge: Structural Analysis and Optimization of Convolutional Neural Networks with a Small Sample Size

2018 ◽  
Author(s):  
Rhett N. D’souza ◽  
Po-Yao Huang ◽  
Fang-Cheng Yeh
Keyword(s):  
Neural Networks ◽  
Big Data ◽  
Sample Size ◽  
Network Structure ◽  
Small Sample Size ◽  
Small Sample ◽  
Small Data ◽  
Optimal Network ◽  
Training Samples ◽  
Improved Accuracy

AbstractDeep neural networks have gained immense popularity in the Big Data problem; however, the availability of training samples can be relatively limited in certain application domains, particularly medical imaging, and consequently leading to overfitting problems. This “Small Data” challenge may need a mindset that is entirely different from the existing Big Data paradigm. Here, under the small data setting, we examined whether the network structure has a substantial influence on the performance and whether the optimal structure is predominantly determined by sample size or data nature. To this end, we listed all possible combinations of layers given an upper bound of the VC-dimension to study how structural hyperparameters affected the performance. Our results showed that structural optimization improved accuracy by 27.99%, 16.44%, and 13.11% over random selection for a sample size of 100, 500, and 1,000 in the MNIST dataset, respectively, suggesting that the importance of the network structure increases as the sample size becomes smaller. Furthermore, the optimal network structure was mostly determined by the data nature (photographic, calligraphic, or medical images), and less affected by the sample size, suggesting that the optimal network structure is data-driven, not sample size driven. After network structure optimization, the conventional convolutional neural network could achieve 91.13% in accuracy with only 500 samples, 93.66% in accuracy with only 1000 samples for the MNIST dataset and 94.10% in accuracy with only 3300 samples for the Mitosis (microscopic) dataset. These results indicate the primary importance of the network structure and the nature of the data in facing the Small Data challenge.

Transfer learning with convolutional neural networks for small sample size problem in machinery fault diagnosis

2019 ◽  
Vol 233 (14) ◽  
pp. 5131-5143 ◽  
Author(s):  
Dengyu Xiao ◽  
Yixiang Huang ◽  
Chengjin Qin ◽  
Zhiyu Liu ◽  
Yanming Li ◽  
...  
Keyword(s):  
Neural Networks ◽  
Fault Diagnosis ◽  
Sample Size ◽  
Transfer Learning ◽  
Small Sample Size ◽  
Small Sample ◽  
Diagnostic Model ◽  
Source Data ◽  
Target Data ◽  
Machinery Fault Diagnosis

Data-driven machinery fault diagnosis has gained much attention from academic research and industry to guarantee the machinery reliability. Traditional fault diagnosis frameworks are commonly under a default assumption: the training and test samples share the similar distribution. However, it is nearly impossible in real industrial applications, where the operating condition always changes over time and the quantity of the same-distribution samples is often not sufficient to build a qualified diagnostic model. Therefore, transfer learning, which possesses the capacity to leverage the knowledge learnt from the massive source data to establish a diagnosis model for the similar but small target data, has shown potential value in machine fault diagnosis with small sample size. In this paper, we propose a novel fault diagnosis framework for the small amount of target data based on transfer learning, using a modified TrAdaBoost algorithm and convolutional neural networks. First, the massive source data with different distributions is added to the target data as the training data. Then, a convolutional neural network is selected as the base learner and the modified TrAdaBoost algorithm is employed for the weight update of each training sample to form a stronger diagnostic model. The whole proposition is experimentally demonstrated and discussed by carrying out the tests of six three-phase induction motors under different operating conditions and fault types. Results show that compared with other methods, the proposed framework can achieve the highest fault diagnostic accuracy with inadequate target data.

scholarly journals The Classification Between Prostate Transitional Zone Cancer and Hyperplasia Using Deep Transfer Learning from Disease-Related Images: A Retrospective Study

2020 ◽  
Author(s):  
Bo Hu ◽  
Lin-Feng Yan ◽  
Yang Yang ◽  
Ying-Zhi Sun ◽  
Cui Yue ◽  
...  
Keyword(s):  
Sample Size ◽  
Transfer Learning ◽  
Small Sample Size ◽  
Transitional Zone ◽  
Small Sample ◽  
Fine Tuning ◽  
Small Data ◽  
Deep Convolutional Neural Networks ◽  
Tuning Method ◽  
Apparent Diffusion

Abstract Background The diagnosis of prostate transition zone cancers (PTZC) remains a clinical challenge due to its similarity to benign prostatic hyperplasia (BPH) on MRI. The Deep Convolutional Neural Networks showed high efficacy in medical imaging but was limited by the small data size. A transfer learning method was combined with deep learning to overcome this challenge.Methods A retrospective investigation was conducted on 217 patients enrolled from our hospital database (208 patients) and The Cancer Imaging Archive (9 patients). Based on the T2 weighted images (T2WIs) and apparent diffusion coefficient (ADC) maps of these patients, DCNN models were trained and compared between different TL database (ImageNet vs. disease-related images) and protocols (from scratch, fine-tuning or transductive transferring).Results PTZC and BPH can be classified through traditional DCNN. The efficacy of transfer learning from ImageNet was limited but improved by transferring knowledge from the disease-related images. Furthermore, transductive transfer learning from disease-related images had the comparable efficacies with the fine-tuning method. Limitations include retrospective design and relatively small sample size.Conclusion For PTZC with a small sample size, the accurate diagnosis can be achieved via the deep transfer learning from disease-related images.

scholarly journals Identification of cucumber leaf diseases using deep learning and small sample size for agricultural Internet of Things

2021 ◽  
Vol 17 (4) ◽  
pp. 155014772110074
Author(s):  
Jingyao Zhang ◽  
Yuan Rao ◽  
Chao Man ◽  
Zhaohui Jiang ◽  
Shaowen Li
Keyword(s):  
Neural Network ◽  
Internet Of Things ◽  
Convolutional Neural Network ◽  
Sample Size ◽  
Data Augmentation ◽  
Small Sample Size ◽  
Small Sample ◽  
Identification Accuracy ◽  
Generative Adversarial Networks ◽  
Training Samples

Due to the complex environments in real fields, it is challenging to conduct identification modeling and diagnosis of plant leaf diseases by directly utilizing in-situ images from the system of agricultural Internet of things. To overcome this shortcoming, one approach, based on small sample size and deep convolutional neural network, was proposed for conducting the recognition of cucumber leaf diseases under field conditions. One two-stage segmentation method was presented to acquire the lesion images by extracting the disease spots from cucumber leaves. Subsequently, after implementing rotation and translation, the lesion images were fed into the activation reconstruction generative adversarial networks for data augmentation to generate new training samples. Finally, to improve the identification accuracy of cucumber leaf diseases, we proposed dilated and inception convolutional neural network that was trained using the generated training samples. Experimental results showed that the proposed approach achieved the average identification accuracy of 96.11% and 90.67% when implemented on the data sets of lesion and raw field diseased leaf images with three different diseases of anthracnose, downy mildew, and powdery mildew, significantly outperforming those existing counterparts, indicating that it offered good potential of serving field application of agricultural Internet of things.

scholarly journals Big data approaches to decomposing heterogeneity across the autism spectrum

2018 ◽  
Author(s):  
Michael V. Lombardo ◽  
Meng-Chuan Lai ◽  
Simon Baron-Cohen
Keyword(s):  
Big Data ◽  
Sample Size ◽  
Small Sample Size ◽  
Autism Spectrum ◽  
Small Sample ◽  
Levels Of Analysis ◽  
Large Sample Size ◽  
Autistic People ◽  
Multi Level ◽  
Multiple Levels

AbstractAutism is a diagnostic label based on behavior. While the diagnostic criteria attempts to maximize clinical consensus, it also masks a wide degree of heterogeneity between and within individuals at multiple levels of analysis. Understanding this multi-level heterogeneity is of high clinical and translational importance. Here we present organizing principles to frame the work examining multi-level heterogeneity in autism. Theoretical concepts such as ‘spectrum’ or ‘autisms’ reflect non-mutually exclusive explanations regarding continuous/dimensional or categorical/qualitative variation between and within individuals. However, common practices of small sample size studies and case-control models are suboptimal for tackling heterogeneity. Big data is an important ingredient for furthering our understanding heterogeneity in autism. In addition to being ‘feature-rich’, big data should be both ‘broad’ (i.e. large sample size) and ‘deep’ (i.e. multiple levels of data collected on the same individuals). These characteristics help ensure the results from a population are more generalizable and facilitate evaluation of the utility of different models of heterogeneity. A model’s utility can be shown by its ability to explain clinically or mechanistically important phenomena, but also by explaining how variability manifests across different levels of analysis. The directionality for explaining variability across levels can be bottom-up or top-down, and should include the importance of development for characterizing change within individuals. While progress can be made with ‘supervised’ models built upon a priori or theoretically predicted distinctions or dimensions of importance, it will become increasingly important to complement such work with unsupervised data-driven discoveries that leverage unknown and multivariate distinctions within big data. Without a better understanding of how to model heterogeneity between autistic people, progress towards the goal of precision medicine may be limited.

Application of Ultrasound Elastography for Assessing Intestinal Fibrosis in Inflammatory Bowel Disease: Fiction or Reality?

2020 ◽  
Vol 21 ◽  
Author(s):  
Roberto Gabbiadini ◽  
Eirini Zacharopoulou ◽  
Federica Furfaro ◽  
Vincenzo Craviotto ◽  
Alessandra Zilli ◽  
...  
Keyword(s):  
Inflammatory Bowel Disease ◽  
Sample Size ◽  
Bowel Disease ◽  
Small Sample Size ◽  
Shear Wave Elastography ◽  
Small Sample ◽  
Ultrasound Elastography ◽  
Invasive Technique ◽  
Intestinal Fibrosis ◽  
Inflammatory Bowel

Background: Intestinal fibrosis and subsequent strictures represent an important burden in inflammatory bowel disease (IBD). The detection and evaluation of the degree of fibrosis in stricturing Crohn’s disease (CD) is important to address the best therapeutic strategy (medical anti-inflammatory therapy, endoscopic dilation, surgery). Ultrasound elastography (USE) is a non-invasive technique that has been proposed in the field of IBD for evaluating intestinal stiffness as a biomarker of intestinal fibrosis. Objective: The aim of this review is to discuss the ability and current role of ultrasound elastography in the assessment of intestinal fibrosis. Results and Conclusion: Data on USE in IBD are provided by pilot and proof-of-concept studies with small sample size. The first type of USE investigated was strain elastography, while shear wave elastography has been introduced lately. Despite the heterogeneity of the methods of the studies, USE has been proven to be able to assess intestinal fibrosis in patients with stricturing CD. However, before introducing this technique in current practice, further studies with larger sample size and homogeneous parameters, testing reproducibility, and identification of validated cut-off values are needed.

Sign in / Sign up

Export Citation Format

Share Document