Unsupervised classification of snowflake images using a generative adversarial network and <i>K</i>-medoids classification

Automated Classification ◽

Microphysical Properties ◽

Adversarial Network ◽

Computationally Intensive ◽

Comparison Of The Results ◽

Supervised Classification Methods

Abstract. The increasing availability of sensors imaging cloud and precipitation particles, like the Multi-Angle Snowflake Camera (MASC), has resulted in datasets comprising millions of images of falling snowflakes. Automated classification is required for effective analysis of such large datasets. While supervised classification methods have been developed for this purpose in the recent years, their ability to generalize is limited by the representativeness of their labeled training datasets, which are affected by the subjective judgment of the expert and require significant manual effort to derive. An alternative is unsupervised classification, which seeks to divide a dataset into distinct classes without expert-provided labels. In this study, we introduce an unsupervised classification scheme based on a generative adversarial network (GAN) that learns to extract the key features from the images. Each image is then associated with a distribution of points in the feature space, and these distributions are used as the basis of K-medoids classification and hierarchical clustering. We find that the classification scheme is able to separate the dataset into distinct classes, each characterized by a particular size, shape and texture of the snowflake image, providing signatures of the microphysical properties of the snowflakes. This finding is supported by a comparison of the results to an existing supervised scheme. Although training the GAN is computationally intensive, the classification process proceeds directly from images to classes with minimal human intervention and therefore can be repeated for other MASC datasets with minor manual effort. As the algorithm is not specific to snowflakes, we also expect this approach to be relevant to other applications.

Unsupervised classification of snowflake images using a generative adversarial network and <i>K</i>-medoids classification

Atmospheric Measurement Techniques ◽

10.5194/amt-13-2949-2020 ◽

2020 ◽

Vol 13 (6) ◽

pp. 2949-2964

Author(s):

Jussi Leinonen ◽

Alexis Berne

Keyword(s):

Classification Scheme ◽

Feature Space ◽

Automated Classification ◽

Microphysical Properties ◽

Adversarial Network ◽

Computationally Intensive ◽

Comparison Of The Results ◽

Supervised Classification Methods

Abstract. The increasing availability of sensors imaging cloud and precipitation particles, like the Multi-Angle Snowflake Camera (MASC), has resulted in datasets comprising millions of images of falling snowflakes. Automated classification is required for effective analysis of such large datasets. While supervised classification methods have been developed for this purpose in recent years, their ability to generalize is limited by the representativeness of their labeled training datasets, which are affected by the subjective judgment of the expert and require significant manual effort to derive. An alternative is unsupervised classification, which seeks to divide a dataset into distinct classes without expert-provided labels. In this paper, we introduce an unsupervised classification scheme based on a generative adversarial network (GAN) that learns to extract the key features from the snowflake images. Each image is then associated with a distribution of points in the feature space, and these distributions are used as the basis of K-medoids classification and hierarchical clustering. We found that the classification scheme is able to separate the dataset into distinct classes, each characterized by a particular size, shape and texture of the snowflake image, providing signatures of the microphysical properties of the snowflakes. This finding is supported by a comparison of the results to an existing supervised scheme. Although training the GAN is computationally intensive, the classification process proceeds directly from images to classes with minimal human intervention and therefore can be repeated for other MASC datasets with minor manual effort. As the algorithm is not specific to snowflakes, we also expect this approach to be relevant to other applications.

Semi Supervised Generative Adversarial Network for Automated Glaucoma Diagnosis with Stacked Discriminator Models

Journal of Medical Imaging and Health Informatics ◽

10.1166/jmihi.2021.3787 ◽

2021 ◽

Vol 11 (5) ◽

pp. 1334-1340

Author(s):

K. Gokul Kannan ◽

T. R. Ganesh Babu

Keyword(s):

Network Architecture ◽

Super Resolution ◽

Learning Task ◽

Generative Models ◽

Adversarial Network ◽

Glaucoma Diagnosis ◽

Image Translation

Generative Adversarial Network (GAN) is neural network architecture, widely used in many computer vision applications such as super-resolution image generation, art creation and image to image translation. A conventional GAN model consists of two sub-models; generative model and discriminative model. The former one generates new samples based on an unsupervised learning task, and the later one classifies them into real or fake. Though GAN is most commonly used for training generative models, it can be used for developing a classifier model. The main objective is to extend the effectiveness of GAN into semi-supervised learning, i.e., for the classification of fundus images to diagnose glaucoma. The discriminator model in the conventional GAN is improved via transfer learning to predict n + 1 classes by training the model for both supervised classification (n classes) and unsupervised classification (fake or real). Both models share all feature extraction layers and differ in the output layers. Thus any update in one of the model will impact both models. Results show that the semi-supervised GAN performs well than a standalone Convolution Neural Networks (CNNs) model.

Feature Learning for SAR Target Recognition with Unknown Classes by Using CVAE-GAN

Remote Sensing ◽

10.3390/rs13183554 ◽

2021 ◽

Vol 13 (18) ◽

pp. 3554

Author(s):

Xiaowei Hu ◽

Weike Feng ◽

Yiduo Guo ◽

Qiang Wang

Keyword(s):

Target Recognition ◽

Feature Learning ◽

Feature Space ◽

Automatic Target Recognition ◽

Data Sets ◽

Data Set ◽

Adversarial Network ◽

Public Data ◽

Class Labels

Even though deep learning (DL) has achieved excellent results on some public data sets for synthetic aperture radar (SAR) automatic target recognition(ATR), several problems exist at present. One is the lack of transparency and interpretability for most of the existing DL networks. Another is the neglect of unknown target classes which are often present in practice. To solve the above problems, a deep generation as well as recognition model is derived based on Conditional Variational Auto-encoder (CVAE) and Generative Adversarial Network (GAN). A feature space for SAR-ATR is built based on the proposed CVAE-GAN model. By using the feature space, clear SAR images can be generated with given class labels and observation angles. Besides, the feature of the SAR image is continuous in the feature space and can represent some attributes of the target. Furthermore, it is possible to classify the known classes and reject the unknown target classes by using the feature space. Experiments on the MSTAR data set validate the advantages of the proposed method.

Category-Sensitive Domain Adaptation for Land Cover Mapping in Aerial Scenes

Remote Sensing ◽

10.3390/rs11222631 ◽

2019 ◽

Vol 11 (22) ◽

pp. 2631 ◽

Cited By ~ 2

Author(s):

Bo Fang ◽

Rong Kou ◽

Li Pan ◽

Pengfei Chen

Keyword(s):

Land Cover ◽

Domain Adaptation ◽

Feature Space ◽

Aerial Images ◽

Land Cover Mapping ◽

Target Domain ◽

Source Domain ◽

Adversarial Network ◽

Semantic Labeling

Since manually labeling aerial images for pixel-level classification is expensive and time-consuming, developing strategies for land cover mapping without reference labels is essential and meaningful. As an efficient solution for this issue, domain adaptation has been widely utilized in numerous semantic labeling-based applications. However, current approaches generally pursue the marginal distribution alignment between the source and target features and ignore the category-level alignment. Therefore, directly applying them to land cover mapping leads to unsatisfactory performance in the target domain. In our research, to address this problem, we embed a geometry-consistent generative adversarial network (GcGAN) into a co-training adversarial learning network (CtALN), and then develop a category-sensitive domain adaptation (CsDA) method for land cover mapping using very-high-resolution (VHR) optical aerial images. The GcGAN aims to eliminate the domain discrepancies between labeled and unlabeled images while retaining their intrinsic land cover information by translating the features of the labeled images from the source domain to the target domain. Meanwhile, the CtALN aims to learn a semantic labeling model in the target domain with the translated features and corresponding reference labels. By training this hybrid framework, our method learns to distill knowledge from the source domain and transfers it to the target domain, while preserving not only global domain consistency, but also category-level consistency between labeled and unlabeled images in the feature space. The experimental results between two airborne benchmark datasets and the comparison with other state-of-the-art methods verify the robustness and superiority of our proposed CsDA.

Self-Difference Convolutional Neural Network for Facial Expression Recognition

Sensors ◽

10.3390/s21062250 ◽

2021 ◽

Vol 21 (6) ◽

pp. 2250

Author(s):

Leyuan Liu ◽

Rubin Jiang ◽

Jiao Huo ◽

Jingying Chen

Keyword(s):

Facial Expression ◽

Facial Expressions ◽

Facial Expression Recognition ◽

Low Cost ◽

Feature Space ◽

Expression Recognition ◽

Convolutional Network ◽

Online Processing ◽

Adversarial Network

Facial expression recognition (FER) is a challenging problem due to the intra-class variation caused by subject identities. In this paper, a self-difference convolutional network (SD-CNN) is proposed to address the intra-class variation issue in FER. First, the SD-CNN uses a conditional generative adversarial network to generate the six typical facial expressions for the same subject in the testing image. Second, six compact and light-weighted difference-based CNNs, called DiffNets, are designed for classifying facial expressions. Each DiffNet extracts a pair of deep features from the testing image and one of the six synthesized expression images, and compares the difference between the deep feature pair. In this way, any potential facial expression in the testing image has an opportunity to be compared with the synthesized “Self”—an image of the same subject with the same facial expression as the testing image. As most of the self-difference features of the images with the same facial expression gather tightly in the feature space, the intra-class variation issue is significantly alleviated. The proposed SD-CNN is extensively evaluated on two widely-used facial expression datasets: CK+ and Oulu-CASIA. Experimental results demonstrate that the SD-CNN achieves state-of-the-art performance with accuracies of 99.7% on CK+ and 91.3% on Oulu-CASIA, respectively. Moreover, the model size of the online processing part of the SD-CNN is only 9.54 MB (1.59 MB ×6), which enables the SD-CNN to run on low-cost hardware.

Semi-Supervised Multi-Temporal Deep Representation Fusion Network for Landslide Mapping from Aerial Orthophotos

Remote Sensing ◽

10.3390/rs13040548 ◽

2021 ◽

Vol 13 (4) ◽

pp. 548

Author(s):

Xiaokang Zhang ◽

Man-On Pun ◽

Ming Liu

Keyword(s):

Feature Space ◽

Representation Learning ◽

Feature Representation ◽

Adversarial Network ◽

Multi Temporal ◽

Landslide Mapping ◽

Spatio Temporal ◽

Reliability And Robustness ◽

High Level

Using remote sensing techniques to monitor landslides and their resultant land cover changes is fundamentally important for risk assessment and hazard prevention. Despite enormous efforts in developing intelligent landslide mapping (LM) approaches, LM remains challenging owing to high spectral heterogeneity of very-high-resolution (VHR) images and the daunting labeling efforts. To this end, a deep learning model based on semi-supervised multi-temporal deep representation fusion network, namely SMDRF-Net, is proposed for reliable and efficient LM. In comparison with previous methods, the SMDRF-Net possesses three distinct properties. (1) Unsupervised deep representation learning at the pixel- and object-level is performed by transfer learning using the Wasserstein generative adversarial network with gradient penalty to learn discriminative deep features and retain precise outlines of landslide objects in the high-level feature space. (2) Attention-based adaptive fusion of multi-temporal and multi-level deep representations is developed to exploit the spatio-temporal dependencies of deep representations and enhance the feature representation capability of the network. (3) The network is optimized using limited samples with pseudo-labels that are automatically generated based on a comprehensive uncertainty index. Experimental results from the analysis of VHR aerial orthophotos demonstrate the reliability and robustness of the proposed approach for LM in comparison with state-of-the-art methods.

Generative Adversarial Network Synthesis of Hyperspectral Vegetation Data

Remote Sensing ◽

10.3390/rs13122243 ◽

2021 ◽

Vol 13 (12) ◽

pp. 2243

Author(s):

Andrew Hennessy ◽

Kenneth Clarke ◽

Megan Lewis

Keyword(s):

Precision Agriculture ◽

Classification Accuracy ◽

Feature Space ◽

Hyperspectral Data ◽

Network Synthesis ◽

Adversarial Network ◽

Training Samples ◽

Almost All ◽

Outlying Regions

New, accurate and generalizable methods are required to transform the ever-increasing amount of raw hyperspectral data into actionable knowledge for applications such as environmental monitoring and precision agriculture. Here, we apply advances in generative deep learning models to produce realistic synthetic hyperspectral vegetation data, whilst maintaining class relationships. Specifically, a Generative Adversarial Network (GAN) is trained using the Cramér distance on two vegetation hyperspectral datasets, demonstrating the ability to approximate the distribution of the training samples. Evaluation of the synthetic spectra shows that they respect many of the statistical properties of the real spectra, conforming well to the sampled distributions of all real classes. Creation of an augmented dataset consisting of synthetic and original samples was used to train multiple classifiers, with increases in classification accuracy seen under almost all circumstances. Both datasets showed improvements in classification accuracy ranging from a modest 0.16% for the Indian Pines set and a substantial increase of 7.0% for the New Zealand vegetation. Selection of synthetic samples from sparse or outlying regions of the feature space of real spectral classes demonstrated increased discriminatory power over those from more central portions of the distributions.

Dynamic Detection and Recognition of Objects Based on Sequential RGB Images

Future Internet ◽

10.3390/fi13070176 ◽

2021 ◽

Vol 13 (7) ◽

pp. 176

Author(s):

Shuai Dong ◽

Zhihua Yang ◽

Wensheng Li ◽

Kun Zou

Keyword(s):

Level Set ◽

Mapping Function ◽

Feature Space ◽

Retrieval Task ◽

Fine Grained ◽

Adversarial Network ◽

Dynamic Detection ◽

Instance Segmentation ◽

Detection And Recognition

Conveyors are used commonly in industrial production lines and automated sorting systems. Many applications require fast, reliable, and dynamic detection and recognition for the objects on conveyors. Aiming at this goal, we design a framework that involves three subtasks: one-class instance segmentation (OCIS), multiobject tracking (MOT), and zero-shot fine-grained recognition of 3D objects (ZSFGR3D). A new level set map network (LSMNet) and a multiview redundancy-free feature network (MVRFFNet) are proposed for the first and third subtasks, respectively. The level set map (LSM) is used to annotate instances instead of the traditional multichannel binary mask, and each peak of the LSM represents one instance. Based on the LSM, LSMNet can adopt a pix2pix architecture to segment instances. MVRFFNet is a generalized zero-shot learning (GZSL) framework based on the Wasserstein generative adversarial network for 3D object recognition. Multi-view features of an object are combined into a compact registered feature. By treating the registered features as the category attribution in the GZSL setting, MVRFFNet learns a mapping function that maps original retrieve features into a new redundancy-free feature space. To validate the performance of the proposed methods, a segmentation dataset and a fine-grained classification dataset about objects on a conveyor are established. Experimental results on these datasets show that LSMNet can achieve a recalling accuracy close to the light instance segmentation framework You Only Look At CoefficienTs (YOLACT), while its computing speed on an NVIDIA GTX1660TI GPU is 80 fps, which is much faster than YOLACT‘s 25 fps. Redundancy-free features generated by MVRFFNet perform much better than original features in the retrieval task.

A Data-Efficient Approach for Automated Classification of OCT Images Using Generative Adversarial Network

IEEE Sensors Letters ◽

10.1109/lsens.2019.2963712 ◽

2020 ◽

Vol 4 (1) ◽

pp. 1-4

Author(s):

Vineeta Das ◽

Samarendra Dandapat ◽

Prabin Kumar Bora

Keyword(s):

Automated Classification ◽

Efficient Approach ◽

Adversarial Network

An Automatic Classification of the Early Osteonecrosis of Femoral Head with Deep Learning

Current Medical Imaging Formerly Current Medical Imaging Reviews ◽

10.2174/1573405615666191212104639 ◽

2019 ◽

Vol 15 ◽

Author(s):

Liyang Zhu ◽

Jungang Han ◽

Renwen Guo ◽

Dong Wu ◽

Qiang Wei ◽

...

Keyword(s):

Deep Learning ◽

Femoral Head ◽

Femur Head ◽

Osteonecrosis Of Femoral Head ◽

Adversarial Network ◽

Convolutional Autoencoder ◽

Ct Slices

Background: Osteonecrosis of Femoral Head (ONFH) is a common complication in orthopaedics, wherein femoral structures are usually damaged due to the impairment or interruption of femoral head blood supply. Aim: In this study, we propose an automatic approach for the classification of the early ONFH with deep learning. Methods: We first classify all femoral CT slices according to their spatial locations with the Convolutional Neural Network (CNN). So, all CT slices are divided into upper, middle or lower segments of femur head. Then the femur head areas can be segmented with the Conditional Generative Adversarial Network (CGAN) for each part. The Convolutional Autoencoder is employed to reduce dimensions and extract features of femur head, and finally K-means clustering is used for an unsupervised classification of the early ONFH. Results: To invalidate the effectiveness of the proposed approach, we carry out the experiments on the dataset with 120 patients. The experimental results show that the segmentation accuracy is higher than 95%. The Convolutional Autoencoder can reduce the dimension of data, the Peak Signal-to-Noise Ratios (PSNRs) are better than 34dB for inputs and outputs. Meanwhile, there is a great intra-category similarity, and a significant inter-category difference. Conclusion: The research on the classification of the early ONFH has a valuable clinical merit, and hopefully it can assist physicians to apply more individualized treatment for patient.