scholarly journals Learning Student Networks with Few Data

2020 ◽  
Vol 34 (04) ◽  
pp. 4469-4476 ◽  
Author(s):  
Shumin Kong ◽  
Tianyu Guo ◽  
Shan You ◽  
Chang Xu
Keyword(s):  
Data Privacy ◽  
Empirical Distribution ◽  
Ground Truth ◽  
Wasserstein Distance ◽  
Worst Case ◽  
Ground Truth Data ◽  
Teacher Student ◽  
Benchmark Datasets ◽  
Training Examples ◽  
Few Data

Recently, the teacher-student learning paradigm has drawn much attention in compressing neural networks on low-end edge devices, such as mobile phones and wearable watches. Current algorithms mainly assume the complete dataset for the teacher network is also available for the training of the student network. However, for real-world scenarios, users may only have access to part of training examples due to commercial profits or data privacy, and severe over-fitting issues would happen as a result. In this paper, we tackle the challenge of learning student networks with few data by investigating the ground-truth data-generating distribution underlying these few data. Taking Wasserstein distance as the measurement, we assume this ideal data distribution lies in a neighborhood of the discrete empirical distribution induced by the training examples. Thus we propose to safely optimize the worst-case cost within this neighborhood to boost the generalization. Furthermore, with theoretical analysis, we derive a novel and easy-to-implement loss for training the student network in an end-to-end fashion. Experimental results on benchmark datasets validate the effectiveness of our proposed method.

scholarly journals Retinal blood vessel segmentation from retinal image using B-COSFIRE and adaptive thresholding

2019 ◽  
Vol 13 (3) ◽  
pp. 1199 ◽  
Author(s):  
Aziah Ali ◽  
Wan Mimi Diyana Wan Zaki ◽  
Aini Hussain
Keyword(s):  
Retinal Image ◽  
Ground Truth ◽  
Threshold Value ◽  
Adaptive Thresholding ◽  
Computer Assisted ◽  
Segmentation Method ◽  
Ground Truth Data ◽  
Segmentation Methods ◽  
Otsu’S Method ◽  
Benchmark Datasets

<span>Segmentation of blood vessels (BVs) from retinal image is one of the important steps in developing a computer-assisted retinal diagnosis system and has been widely researched especially for implementing automatic BV segmentation methods. This paper proposes an improvement to an existing retinal BV (RBV) segmentation method by combining the trainable B-COSFIRE filter with adaptive thresholding methods. The proposed method can automatically configure its selectivity given a prototype pattern to be detected. Its segmentation performance is comparable to many published methods with the advantage of robustness against noise on retinal background. Instead of using grid search to find the optimal threshold value for a whole dataset, adaptive thresholding (AT) is used to determine the threshold for each retinal image. Two AT methods investigated in this study were ISODATA and Otsu’s method. The proposed method was validated using 40 images from two benchmark datasets for retinal BV segmentation validation, namely DRIVE and STARE. The validation results indicated that the segmentation performance of the proposed unsupervised method is comparable to the original B-COSFIRE method and other published methods, without requiring the availability of ground truth data for new dataset. The Sensitivity and Specificity values achieved for DRIVE and STARE are 0.7818, 0.9688, 0.7957 and 0.9648, respectively.</span>

scholarly journals Increasing the Effectiveness of Active Learning: Introducing Artificial Data Generation in Active Learning for Land Use/Land Cover Classification

2021 ◽  
Vol 13 (13) ◽  
pp. 2619
Author(s):  
Joao Fonseca ◽  
Georgios Douzas ◽  
Fernando Bacao
Keyword(s):  
Active Learning ◽  
Performance Metrics ◽  
User Interaction ◽  
Ground Truth ◽  
Data Generation ◽  
Artificial Data ◽  
Ground Truth Data ◽  
Data Generator ◽  
Benchmark Datasets ◽  
Time Requirements

In remote sensing, Active Learning (AL) has become an important technique to collect informative ground truth data ``on-demand'' for supervised classification tasks. Despite its effectiveness, it is still significantly reliant on user interaction, which makes it both expensive and time consuming to implement. Most of the current literature focuses on the optimization of AL by modifying the selection criteria and the classifiers used. Although improvements in these areas will result in more effective data collection, the use of artificial data sources to reduce human--computer interaction remains unexplored. In this paper, we introduce a new component to the typical AL framework, the data generator, a source of artificial data to reduce the amount of user-labeled data required in AL. The implementation of the proposed AL framework is done using Geometric SMOTE as the data generator. We compare the new AL framework to the original one using similar acquisition functions and classifiers over three AL-specific performance metrics in seven benchmark datasets. We show that this modification of the AL framework significantly reduces cost and time requirements for a successful AL implementation in all of the datasets used in the experiment.

scholarly journals A NEW STEREO DENSE MATCHING BENCHMARK DATASET FOR DEEP LEARNING

2021 ◽  
Vol XLIII-B2-2021 ◽  
pp. 405-412
Author(s):  
T. Wu ◽  
B. Vallet ◽  
M. Pierrot-Deseilligny ◽  
E. Rupnik
Keyword(s):  
Deep Learning ◽  
Ground Truth ◽  
Fine Tuning ◽  
Training Dataset ◽  
Ground Truth Data ◽  
Aerial Photogrammetry ◽  
Dense Matching ◽  
Benchmark Datasets ◽  
3D Scene ◽  
The Impact

Abstract. Stereo dense matching is a fundamental task for 3D scene reconstruction. Recently, deep learning based methods have proven effective on some benchmark datasets, for example Middlebury and KITTI stereo. However, it is not easy to find a training dataset for aerial photogrammetry. Generating ground truth data for real scenes is a challenging task. In the photogrammetry community, many evaluation methods use digital surface models (DSM) to generate the ground truth disparity for the stereo pairs, but in this case interpolation may bring errors in the estimated disparity. In this paper, we publish a stereo dense matching dataset based on ISPRS Vaihingen dataset, and use it to evaluate some traditional and deep learning based methods. The evaluation shows that learning-based methods outperform traditional methods significantly when the fine tuning is done on a similar landscape. The benchmark also investigates the impact of the base to height ratio on the performance of the evaluated methods. The dataset can be found in https://github.com/whuwuteng/benchmark_ISPRS2021.

scholarly journals Bridging the Gap between Few-Shot and Many-Shot Learning via Distribution Calibration

2021 ◽  
Author(s):  
Shuo Yang ◽  
Songhua Wu ◽  
Tongliang Liu ◽  
Min Xu
Keyword(s):  
Error Bound ◽  
Estimation Error ◽  
Data Distribution ◽  
Approximation Error ◽  
Ground Truth ◽  
Generalization Error ◽  
Learning To Learn ◽  
Ground Truth Data ◽  
Generalization Error Bound ◽  
Training Examples

A major gap between few-shot and many-shot learning is the data distribution empirically observed by the model during training. In few-shot learning, the learned model can easily become over-fitted based on the biased distribution formed by only a few training examples, while the ground-truth data distribution is more accurately uncovered in many-shot learning to learn a well-generalized model. In this paper, we propose to calibrate the distribution of these few-sample classes to be more unbiased to alleviate such an over-fitting problem. The distribution calibration is achieved by transferring statistics from the classes with sufficient examples to those few-sample classes. After calibration, an adequate number of examples can be sampled from the calibrated distribution to expand the inputs to the classifier. Extensive experiments on three datasets, miniImageNet, tieredImageNet, and CUB, show that a simple linear classifier trained using the features sampled from our calibrated distribution can outperform the state-of-the-art accuracy by a large margin. We also establish a generalization error bound for the proposed distribution-calibration-based few-shot learning, which consists of the distribution assumption error, the distribution approximation error, and the estimation error. This generalization error bound theoretically justifies the effectiveness of the proposed method.

scholarly journals ESTIMATING SLOPE FROM RASTER DATA – A TEST OF EIGHT ALGORITHMS AT DIFFERENT RESOLUTIONS IN FLAT AND STEEP TERRAIN

2013 ◽  
Vol 39 (2) ◽  
pp. 41-52 ◽  
Author(s):  
Jing Tang ◽  
Petter Pilesjö ◽  
Andreas Persson
Keyword(s):  
Grid Cell ◽  
Ground Truth ◽  
The Other ◽  
Worst Case ◽  
Ground Truth Data ◽  
Significance Level ◽  
Smoothing Effects ◽  
Quadratic Surface ◽  
Steep Terrain ◽  
Simple Difference

Different slope algorithms can result in totally different estimates. In the worst case, this may lead to inappropriate and useless modelling estimates. A frequent lack of awareness when choosing algorithms justifies a thorough comparison of their characteristics, making it possible for researchers to select an algorithm which is optimal for their purpose. In this study, eight frequently used slope algorithms applied to Digital Elevation Models (DEMs) are compared. The influences of the resolution of the DEM (0.5, 1, 2, and 5 metres), as well as the terrain form (flat and steep terrain), are considered. It should be noted that the focus of the study is not to compare estimates with ‘ground truth’ data, but on the comparisons between the algorithms, and the ways in which they might differ depending on resolution and terrain. Descriptive statistics are calculated in order to characterize the general characteristics of the eight tested algorithms. Eight combinations of DEM resolution and terrain form are analysed. The results show that the Maximum and Simple Difference algorithms always yield higher mean slope values than the other algorithms, while the Constrained Quadratic Surface algorithm produces the lowest values compared to the others. It is concluded that the estimated slope values are heavily dependent on the number of neighbouring cells included in the estimation. An Analysis of Variance (ANOVA) of estimated slope values strongly indicates (at the significance level of 0.01) that the tested algorithms yield statistically different results. The eight algorithms produce different estimates for all tested resolutions and terrain forms but one. The differences are more pronounced in steep terrain and at a higher resolution. More detailed pairwise comparisons between estimated slope values are also carried out. It is concluded that the smoothing effects associated with the Constrained Quadratic Surface algorithm are greater in steeper terrain, showing significantly lower estimates than other algorithms. On the other hand, the Maximum and Simple Difference algorithms show significantly higher estimates in almost all cases, except the combination of steep terrain and low resolution. With an increase in grid cell size, the loss of information contents in DEMs leads to lower estimated slope values as well as smaller relative differences between algorithms. Based on the results of this study it is concluded that the choice of algorithm results in different estimated slope values, and that resolution and terrain influences these differences significantly.

scholarly journals Bridging the Gap between Few-Shot and Many-Shot Learning via Distribution Calibration

2021 ◽  
Author(s):  
Shuo Yang ◽  
Songhua Wu ◽  
Tongliang Liu ◽  
Min Xu
Keyword(s):  
Error Bound ◽  
Estimation Error ◽  
Data Distribution ◽  
Approximation Error ◽  
Ground Truth ◽  
Generalization Error ◽  
Learning To Learn ◽  
Ground Truth Data ◽  
Generalization Error Bound ◽  
Training Examples

A major gap between few-shot and many-shot learning is the data distribution empirically observed by the model during training. In few-shot learning, the learned model can easily become over-fitted based on the biased distribution formed by only a few training examples, while the ground-truth data distribution is more accurately uncovered in many-shot learning to learn a well-generalized model. In this paper, we propose to calibrate the distribution of these few-sample classes to be more unbiased to alleviate such an over-fitting problem. The distribution calibration is achieved by transferring statistics from the classes with sufficient examples to those few-sample classes. After calibration, an adequate number of examples can be sampled from the calibrated distribution to expand the inputs to the classifier. Extensive experiments on three datasets, miniImageNet, tieredImageNet, and CUB, show that a simple linear classifier trained using the features sampled from our calibrated distribution can outperform the state-of-the-art accuracy by a large margin. We also establish a generalization error bound for the proposed distribution-calibration-based few-shot learning, which consists of the distribution assumption error, the distribution approximation error, and the estimation error. This generalization error bound theoretically justifies the effectiveness of the proposed method.

scholarly journals Assessing Wildfire Burn Severity and Its Relationship with Environmental Factors: A Case Study in Interior Alaska Boreal Forest

2021 ◽  
Vol 13 (10) ◽  
pp. 1966
Author(s):  
Christopher W Smith ◽  
Santosh K Panda ◽  
Uma S Bhatt ◽  
Franz J Meyer ◽  
Anushree Badola ◽  
...  
Keyword(s):  
Boreal Forest ◽  
Ground Truth ◽  
Burn Severity ◽  
Classification Methods ◽  
Spectral Indices ◽  
Ground Truth Data ◽  
Burn Scar ◽  
Interior Alaska ◽  
Remote Sensing Methods ◽  
The Relationship

In recent years, there have been rapid improvements in both remote sensing methods and satellite image availability that have the potential to massively improve burn severity assessments of the Alaskan boreal forest. In this study, we utilized recent pre- and post-fire Sentinel-2 satellite imagery of the 2019 Nugget Creek and Shovel Creek burn scars located in Interior Alaska to both assess burn severity across the burn scars and test the effectiveness of several remote sensing methods for generating accurate map products: Normalized Difference Vegetation Index (NDVI), Normalized Burn Ratio (NBR), and Random Forest (RF) and Support Vector Machine (SVM) supervised classification. We used 52 Composite Burn Index (CBI) plots from the Shovel Creek burn scar and 28 from the Nugget Creek burn scar for training classifiers and product validation. For the Shovel Creek burn scar, the RF and SVM machine learning (ML) classification methods outperformed the traditional spectral indices that use linear regression to separate burn severity classes (RF and SVM accuracy, 83.33%, versus NBR accuracy, 73.08%). However, for the Nugget Creek burn scar, the NDVI product (accuracy: 96%) outperformed the other indices and ML classifiers. In this study, we demonstrated that when sufficient ground truth data is available, the ML classifiers can be very effective for reliable mapping of burn severity in the Alaskan boreal forest. Since the performance of ML classifiers are dependent on the quantity of ground truth data, when sufficient ground truth data is available, the ML classification methods would be better at assessing burn severity, whereas with limited ground truth data the traditional spectral indices would be better suited. We also looked at the relationship between burn severity, fuel type, and topography (aspect and slope) and found that the relationship is site-dependent.

scholarly journals Automatic Evaluation of Wheat Resistance to Fusarium Head Blight Using Dual Mask-RCNN Deep Learning Frameworks in Computer Vision

2020 ◽  
Vol 13 (1) ◽  
pp. 26
Author(s):  
Wen-Hao Su ◽  
Jiajing Zhang ◽  
Ce Yang ◽  
Rae Page ◽  
Tamas Szinyei ◽  
...  
Keyword(s):  
Fusarium Head Blight ◽  
Ground Truth ◽  
Wheat Breeding ◽  
Head Blight ◽  
Detection Rates ◽  
Ground Truth Data ◽  
Resistant Cultivars ◽  
Feature Pyramid ◽  
Rater Error ◽  
Wheat Lines

In many regions of the world, wheat is vulnerable to severe yield and quality losses from the fungus disease of Fusarium head blight (FHB). The development of resistant cultivars is one means of ameliorating the devastating effects of this disease, but the breeding process requires the evaluation of hundreds of lines each year for reaction to the disease. These field evaluations are laborious, expensive, time-consuming, and are prone to rater error. A phenotyping cart that can quickly capture images of the spikes of wheat lines and their level of FHB infection would greatly benefit wheat breeding programs. In this study, mask region convolutional neural network (Mask-RCNN) allowed for reliable identification of the symptom location and the disease severity of wheat spikes. Within a wheat line planted in the field, color images of individual wheat spikes and their corresponding diseased areas were labeled and segmented into sub-images. Images with annotated spikes and sub-images of individual spikes with labeled diseased areas were used as ground truth data to train Mask-RCNN models for automatic image segmentation of wheat spikes and FHB diseased areas, respectively. The feature pyramid network (FPN) based on ResNet-101 network was used as the backbone of Mask-RCNN for constructing the feature pyramid and extracting features. After generating mask images of wheat spikes from full-size images, Mask-RCNN was performed to predict diseased areas on each individual spike. This protocol enabled the rapid recognition of wheat spikes and diseased areas with the detection rates of 77.76% and 98.81%, respectively. The prediction accuracy of 77.19% was achieved by calculating the ratio of the wheat FHB severity value of prediction over ground truth. This study demonstrates the feasibility of rapidly determining levels of FHB in wheat spikes, which will greatly facilitate the breeding of resistant cultivars.

scholarly journals Classification of Cattle Behaviours Using Neck-Mounted Accelerometer-Equipped Collars and Convolutional Neural Networks

Sensors ◽  
2021 ◽  
Vol 21 (12) ◽  
pp. 4050
Author(s):  
Dejan Pavlovic ◽  
Christopher Davison ◽  
Andrew Hamilton ◽  
Oskar Marko ◽  
Robert Atkinson ◽  
...  
Keyword(s):  
Neural Network ◽  
Model Performance ◽  
Ground Truth ◽  
Practical Implementation ◽  
Ground Truth Data ◽  
Battery Lifetime ◽  
Implementation Challenges ◽  
Memory Footprint ◽  
Commercial Farms ◽  
Using Data

Monitoring cattle behaviour is core to the early detection of health and welfare issues and to optimise the fertility of large herds. Accelerometer-based sensor systems that provide activity profiles are now used extensively on commercial farms and have evolved to identify behaviours such as the time spent ruminating and eating at an individual animal level. Acquiring this information at scale is central to informing on-farm management decisions. The paper presents the development of a Convolutional Neural Network (CNN) that classifies cattle behavioural states (`rumination’, `eating’ and `other’) using data generated from neck-mounted accelerometer collars. During three farm trials in the United Kingdom (Easter Howgate Farm, Edinburgh, UK), 18 steers were monitored to provide raw acceleration measurements, with ground truth data provided by muzzle-mounted pressure sensor halters. A range of neural network architectures are explored and rigorous hyper-parameter searches are performed to optimise the network. The computational complexity and memory footprint of CNN models are not readily compatible with deployment on low-power processors which are both memory and energy constrained. Thus, progressive reductions of the CNN were executed with minimal loss of performance in order to address the practical implementation challenges, defining the trade-off between model performance versus computation complexity and memory footprint to permit deployment on micro-controller architectures. The proposed methodology achieves a compression of 14.30 compared to the unpruned architecture but is nevertheless able to accurately classify cattle behaviours with an overall F1 score of 0.82 for both FP32 and FP16 precision while achieving a reasonable battery lifetime in excess of 5.7 years.

A machine learning approach to estimate the strain energy absorption in expanded polystyrene foams

2021 ◽  
pp. 0021955X2110210
Author(s):  
Alejandro E Rodríguez-Sánchez ◽  
Héctor Plascencia-Mora
Keyword(s):  
Neural Network ◽  
Energy Absorption ◽  
Mechanical Energy ◽  
Compressive Loading ◽  
Ground Truth ◽  
Expanded Polystyrene ◽  
Polystyrene Foam ◽  
Stress Strain ◽  
Ground Truth Data ◽  
Expanded Polystyrene Foam

Traditional modeling of mechanical energy absorption due to compressive loadings in expanded polystyrene foams involves mathematical descriptions that are derived from stress/strain continuum mechanics models. Nevertheless, most of those models are either constrained using the strain as the only variable to work at large deformation regimes and usually neglect important parameters for energy absorption properties such as the material density or the rate of the applying load. This work presents a neural-network-based approach that produces models that are capable to map the compressive stress response and energy absorption parameters of an expanded polystyrene foam by considering its deformation, compressive loading rates, and different densities. The models are trained with ground-truth data obtained in compressive tests. Two methods to select neural network architectures are also presented, one of which is based on a Design of Experiments strategy. The results show that it is possible to obtain a single artificial neural networks model that can abstract stress and energy absorption solution spaces for the conditions studied in the material. Additionally, such a model is compared with a phenomenological model, and the results show than the neural network model outperforms it in terms of prediction capabilities, since errors around 2% of experimental data were obtained. In this sense, it is demonstrated that by following the presented approach is possible to obtain a model capable to reproduce compressive polystyrene foam stress/strain data, and consequently, to simulate its energy absorption parameters.

Sign in / Sign up

Export Citation Format

Share Document