Semantic Segmentation of Sorghum Using Hyperspectral Data Identifies Genetic Associations

This study describes the evaluation of a range of approaches to semantic segmentation of hyperspectral images of sorghum plants, classifying each pixel as either nonplant or belonging to one of the three organ types (leaf, stalk, panicle). While many current methods for segmentation focus on separating plant pixels from background, organ-specific segmentation makes it feasible to measure a wider range of plant properties. Manually scored training data for a set of hyperspectral images collected from a sorghum association population was used to train and evaluate a set of supervised classification models. Many algorithms show acceptable accuracy for this classification task. Algorithms trained on sorghum data are able to accurately classify maize leaves and stalks, but fail to accurately classify maize reproductive organs which are not directly equivalent to sorghum panicles. Trait measurements extracted from semantic segmentation of sorghum organs can be used to identify both genes known to be controlling variation in a previously measured phenotypes (e.g., panicle size and plant height) as well as identify signals for genes controlling traits not previously quantified in this population (e.g., stalk/leaf ratio). Organ level semantic segmentation provides opportunities to identify genes controlling variation in a wide range of morphological phenotypes in sorghum, maize, and other related grain crops.

Download Full-text

Cluster-Wise Weighted NMF for Hyperspectral Images Unmixing with Imbalanced Data

Remote Sensing ◽

10.3390/rs13020268 ◽

2021 ◽

Vol 13 (2) ◽

pp. 268

Author(s):

Xiaochen Lv ◽

Wenhong Wang ◽

Hongfu Liu

Keyword(s):

Spatial Information ◽

Hyperspectral Image ◽

Imbalanced Data ◽

Reconstruction Error ◽

Hyperspectral Data ◽

Weight Matrix ◽

Hyperspectral Images ◽

Mixed Data ◽

Sparsity Constraints ◽

Additional Constraints

Hyperspectral unmixing is an important technique for analyzing remote sensing images which aims to obtain a collection of endmembers and their corresponding abundances. In recent years, non-negative matrix factorization (NMF) has received extensive attention due to its good adaptability for mixed data with different degrees. The majority of existing NMF-based unmixing methods are developed by incorporating additional constraints into the standard NMF based on the spectral and spatial information of hyperspectral images. However, they neglect to exploit the nature of imbalanced pixels included in the data, which may cause the pixels mixed with imbalanced endmembers to be ignored, and thus the imbalanced endmembers generally cannot be accurately estimated due to the statistical property of NMF. To exploit the information of imbalanced samples in hyperspectral data during the unmixing procedure, in this paper, a cluster-wise weighted NMF (CW-NMF) method for the unmixing of hyperspectral images with imbalanced data is proposed. Specifically, based on the result of clustering conducted on the hyperspectral image, we construct a weight matrix and introduce it into the model of standard NMF. The proposed weight matrix can provide an appropriate weight value to the reconstruction error between each original pixel and the reconstructed pixel in the unmixing procedure. In this way, the adverse effect of imbalanced samples on the statistical accuracy of NMF is expected to be reduced by assigning larger weight values to the pixels concerning imbalanced endmembers and giving smaller weight values to the pixels mixed by majority endmembers. Besides, we extend the proposed CW-NMF by introducing the sparsity constraints of abundance and graph-based regularization, respectively. The experimental results on both synthetic and real hyperspectral data have been reported, and the effectiveness of our proposed methods has been demonstrated by comparing them with several state-of-the-art methods.

Download Full-text

Unsupervised content-preserving transformation for optical microscopy

Light Science & Applications ◽

10.1038/s41377-021-00484-y ◽

2021 ◽

Vol 10 (1) ◽

Cited By ~ 1

Author(s):

Xinyang Li ◽

Guoxun Zhang ◽

Hui Qiao ◽

Feng Bao ◽

Yue Deng ◽

...

Keyword(s):

Deep Learning ◽

Optical Microscopy ◽

Training Data ◽

Fluorescence Labeling ◽

Imaging Data ◽

Image Transformation ◽

General Applicability ◽

Data Annotation ◽

Biomedical Image ◽

Wide Range

AbstractThe development of deep learning and open access to a substantial collection of imaging data together provide a potential solution for computational image transformation, which is gradually changing the landscape of optical imaging and biomedical research. However, current implementations of deep learning usually operate in a supervised manner, and their reliance on laborious and error-prone data annotation procedures remains a barrier to more general applicability. Here, we propose an unsupervised image transformation to facilitate the utilization of deep learning for optical microscopy, even in some cases in which supervised models cannot be applied. Through the introduction of a saliency constraint, the unsupervised model, named Unsupervised content-preserving Transformation for Optical Microscopy (UTOM), can learn the mapping between two image domains without requiring paired training data while avoiding distortions of the image content. UTOM shows promising performance in a wide range of biomedical image transformation tasks, including in silico histological staining, fluorescence image restoration, and virtual fluorescence labeling. Quantitative evaluations reveal that UTOM achieves stable and high-fidelity image transformations across different imaging conditions and modalities. We anticipate that our framework will encourage a paradigm shift in training neural networks and enable more applications of artificial intelligence in biomedical imaging.

Download Full-text

Diversity of Pod Shape in Pisum

Diversity ◽

10.3390/d13050203 ◽

2021 ◽

Vol 13 (5) ◽

pp. 203

Author(s):

Thomas Henry Noel Ellis ◽

Julie M. I. Hofer ◽

Eleni Vikeli ◽

Michael J. Ambrose ◽

Paola Higuera-Poveda ◽

...

Keyword(s):

Allelic Variation ◽

Size Variation ◽

Width Ratio ◽

Negative Regulators ◽

Wide Range ◽

Yield Determination ◽

Organ Specific ◽

Length Width ◽

End Use ◽

Pod Length

The seed-containing pod is the defining structure of plants in the legume family, yet pods exhibit a wide range of morphological variation. Within a species pod characters are likely to be correlated with reproductive strategy, and within cultivated forms will correspond to aspects of yield determination and/or end use. Here variation in pod size, described as pod length: pod width ratio, has been analyzed in pea germplasm represented by 597 accessions. This pod size variation is discussed with respect to population structure and to known classical pod morphology mutants. Variability of the pod length: width ratio can be explained by allelic variation at two genetic loci that may correspond to organ-specific negative regulators of growth.

Download Full-text

BLAINDER—A Blender AI Add-On for Generation of Semantically Labeled Depth-Sensing Data

Sensors ◽

10.3390/s21062144 ◽

2021 ◽

Vol 21 (6) ◽

pp. 2144

Author(s):

Stefan Reitmann ◽

Lorenzo Neumann ◽

Bernhard Jung

Keyword(s):

Point Clouds ◽

Training Data ◽

Sensor Data ◽

Generation Process ◽

Data Generation ◽

Depth Sensor ◽

Depth Sensing ◽

Depth Sensors ◽

3D Point Clouds ◽

Wide Range

Common Machine-Learning (ML) approaches for scene classification require a large amount of training data. However, for classification of depth sensor data, in contrast to image data, relatively few databases are publicly available and manual generation of semantically labeled 3D point clouds is an even more time-consuming task. To simplify the training data generation process for a wide range of domains, we have developed the BLAINDER add-on package for the open-source 3D modeling software Blender, which enables a largely automated generation of semantically annotated point-cloud data in virtual 3D environments. In this paper, we focus on classical depth-sensing techniques Light Detection and Ranging (LiDAR) and Sound Navigation and Ranging (Sonar). Within the BLAINDER add-on, different depth sensors can be loaded from presets, customized sensors can be implemented and different environmental conditions (e.g., influence of rain, dust) can be simulated. The semantically labeled data can be exported to various 2D and 3D formats and are thus optimized for different ML applications and visualizations. In addition, semantically labeled images can be exported using the rendering functionalities of Blender.

Download Full-text

AT2ES: Simultaneous Atmospheric Transmittance-Temperature-Emissivity Separation Using Online Upper Midwave Infrared Hyperspectral Images

Remote Sensing ◽

10.3390/rs13071249 ◽

2021 ◽

Vol 13 (7) ◽

pp. 1249

Author(s):

Sungho Kim ◽

Jungsub Shin ◽

Sunho Kim

Keyword(s):

Air Temperature ◽

Atmospheric Correction ◽

Hyperspectral Data ◽

Hyperspectral Images ◽

Co2 Absorption ◽

Infrared Band ◽

Data Regression ◽

Midwave Infrared ◽

Novel Method ◽

Atmospheric Transmittance

This paper presents a novel method for atmospheric transmittance-temperature-emissivity separation (AT2ES) using online midwave infrared hyperspectral images. Conventionally, temperature and emissivity separation (TES) is a well-known problem in the remote sensing domain. However, previous approaches use the atmospheric correction process before TES using MODTRAN in the long wave infrared band. Simultaneous online atmospheric transmittance-temperature-emissivity separation starts with approximation of the radiative transfer equation in the upper midwave infrared band. The highest atmospheric band is used to estimate surface temperature, assuming high emissive materials. The lowest atmospheric band (CO2 absorption band) is used to estimate air temperature. Through onsite hyperspectral data regression, atmospheric transmittance is obtained from the y-intercept, and emissivity is separated using the observed radiance, the separated object temperature, the air temperature, and atmospheric transmittance. The advantage with the proposed method is from being the first attempt at simultaneous AT2ES and online separation without any prior knowledge and pre-processing. Midwave Fourier transform infrared (FTIR)-based outdoor experimental results validate the feasibility of the proposed AT2ES method.

Download Full-text

Potential and Limitations of Grasslands α-Diversity Prediction Using Fine-Scale Hyperspectral Imagery

Remote Sensing ◽

10.3390/rs13142649 ◽

2021 ◽

Vol 13 (14) ◽

pp. 2649

Author(s):

Hafiz Ali Imran ◽

Damiano Gianelle ◽

Michele Scotton ◽

Duccio Rocchini ◽

Michele Dalponte ◽

...

Keyword(s):

Species Richness ◽

Well Being ◽

Hyperspectral Data ◽

Hyperspectral Images ◽

Grassland Species ◽

Biodiversity Indices ◽

Grassland Ecosystems ◽

Natural Grasslands ◽

Α Diversity ◽

Diversity Metrics

Plant biodiversity is an important feature of grassland ecosystems, as it is related to the provision of many ecosystem services crucial for the human economy and well-being. Given the importance of grasslands, research has been carried out in recent years on the potential to monitor them with novel remote sensing techniques. In this study, the optical diversity (also called spectral diversity) approach was adopted to check the potential of using high-resolution hyperspectral images to estimate α-diversity in grassland ecosystems. In 2018 and 2019, grassland species composition was surveyed and canopy hyperspectral data were acquired at two grassland sites: Monte Bondone (IT-MBo; species-rich semi-natural grasslands) and an experimental farm of the University of Padova, Legnaro, Padua, Italy (IT-PD; artificially established grassland plots with a species-poor mixture). The relationship between biodiversity (species richness, Shannon’s, species evenness, and Simpson’s indices) and optical diversity metrics (coefficient of variation-CV and standard deviation-SD) was not consistent across the investigated grassland plant communities. Species richness could be estimated by optical diversity metrics with an R = 0.87 at the IT-PD species-poor site. In the more complex and species-rich grasslands at IT-MBo, the estimation of biodiversity indices was more difficult and the optical diversity metrics failed to estimate biodiversity as accurately as in IT-PD probably due to the higher number of species and the strong canopy spatial heterogeneity. Therefore, the results of the study confirmed the ability of spectral proxies to detect grassland α-diversity in man-made grassland ecosystems but highlighted the limitations of the spectral diversity approach to estimate biodiversity when natural grasslands are observed. Nevertheless, at IT-MBo, the optical diversity metric SD calculated from post-processed hyperspectral images and transformed spectra showed, in the red part of the spectrum, a significant correlation (up to R = 0.56, p = 0.004) with biodiversity indices. Spatial resampling highlighted that for the IT-PD sward the optimal optical pixel size was 1 cm, while for the IT-MBo natural grassland it was 1 mm. The random pixel extraction did not improve the performance of the optical diversity metrics at both study sites. Further research is needed to fully understand the links between α-diversity and spectral and biochemical heterogeneity in complex heterogeneous ecosystems, and to assess whether the optical diversity approach can be adopted at the spatial scale to detect β-diversity. Such insights will provide more robust information on the mechanisms linking grassland diversity and optical heterogeneity.

Download Full-text

A Transcriptomic Approach to the Metabolism of Tetrapyrrolic Photosensitizers in a Marine Annelid

Molecules ◽

10.3390/molecules26133924 ◽

2021 ◽

Vol 26 (13) ◽

pp. 3924

Author(s):

Maria Leonor Santos ◽

Mariaelena D’Ambrosio ◽

Ana P. Rodrigo ◽

A. Jorge Parola ◽

Pedro M. Costa

Keyword(s):

Metabolic Pathways ◽

Molecular Networks ◽

Bile Pigments ◽

Rna Seq ◽

The Past ◽

Wide Range ◽

Genes Encoding ◽

Organ Specific ◽

Bright Green ◽

Green Coloration

The past decade has seen growing interest in marine natural pigments for biotechnological applications. One of the most abundant classes of biological pigments is the tetrapyrroles, which are prized targets due their photodynamic properties; porphyrins are the best known examples of this group. Many animal porphyrinoids and other tetrapyrroles are produced through heme metabolic pathways, the best known of which are the bile pigments biliverdin and bilirubin. Eulalia is a marine Polychaeta characterized by its bright green coloration resulting from a remarkably wide range of greenish and yellowish tetrapyrroles, some of which have promising photodynamic properties. The present study combined metabolomics based on HPLC-DAD with RNA-seq transcriptomics to investigate the molecular pathways of porphyrinoid metabolism by comparing the worm’s proboscis and epidermis, which display distinct pigmentation patterns. The results showed that pigments are endogenous and seemingly heme-derived. The worm possesses homologs in both organs for genes encoding enzymes involved in heme metabolism such as ALAD, FECH, UROS, and PPOX. However, the findings also indicate that variants of the canonical enzymes of the heme biosynthesis pathway can be species- and organ-specific. These differences between molecular networks contribute to explain not only the differential pigmentation patterns between organs, but also the worm’s variety of novel endogenous tetrapyrrolic compounds.

Download Full-text

Are radiomics features universally applicable to different organs?

Cancer Imaging ◽

10.1186/s40644-021-00400-y ◽

2021 ◽

Vol 21 (1) ◽

Author(s):

Seung-Hak Lee ◽

Hwan-ho Cho ◽

Junmo Kwon ◽

Ho Yun Lee ◽

Hyunjin Park

Keyword(s):

Risk Groups ◽

Careful Consideration ◽

Distinct Pattern ◽

Common Category ◽

Independent Test ◽

Organ Specific ◽

Tumor Region ◽

Organ Level ◽

Test Sets ◽

Feature Selection Approach

Abstract Background Many studies have successfully identified radiomics features reflecting macroscale tumor features and tumor microenvironment for various organs. There is an increased interest in applying these radiomics features found in a given organ to other organs. Here, we explored whether common radiomics features could be identified over target organs in vastly different environments. Methods Four datasets of three organs were analyzed. One radiomics model was constructed from the training set (lungs, n = 401), and was further evaluated in three independent test sets spanning three organs (lungs, n = 59; kidneys, n = 48; and brains, n = 43). Intensity histograms derived from the whole organ were compared to establish organ-level differences. We constructed a radiomics score based on selected features using training lung data over the tumor region. A total of 143 features were computed for each tumor. We adopted a feature selection approach that favored stable features, which can also capture survival. The radiomics score was applied to three independent test data from lung, kidney, and brain tumors, and whether the score could be used to separate high- and low-risk groups, was evaluated. Results Each organ showed a distinct pattern in the histogram and the derived parameters (mean and median) at the organ-level. The radiomics score trained from the lung data of the tumor region included seven features, and the score was only effective in stratifying survival for other lung data, not in other organs such as the kidney and brain. Eliminating the lung-specific feature (2.5 percentile) from the radiomics score led to similar results. There were no common features between training and test sets, but a common category of features (texture category) was identified. Conclusion Although the possibility of a generally applicable model cannot be excluded, we suggest that radiomics score models for survival were mostly specific for a given organ; applying them to other organs would require careful consideration of organ-specific properties.

Download Full-text

Optimal Scale of Hierarchical Image Segmentation with Scribbles Guidance for Weakly Supervised Semantic Segmentation

International Journal of Pattern Recognition and Artificial Intelligence ◽

10.1142/s0218001421540264 ◽

2021 ◽

pp. 2154026

Author(s):

Zaid Al-Huda ◽

Donghai Zhai ◽

Yan Yang ◽

Riyadh Nazar Ali Algburi

Keyword(s):

Image Segmentation ◽

Graphical Model ◽

Semantic Segmentation ◽

Saliency Map ◽

Training Data ◽

Deep Convolutional Neural Networks ◽

High Quality ◽

Optimal Scale ◽

Supervised Segmentation ◽

Weakly Supervised

Deep convolutional neural networks (DCNNs) trained on the pixel-level annotated images have achieved improvements in semantic segmentation. Due to the high cost of labeling training data, their applications may have great limitation. However, weakly supervised segmentation approaches can significantly reduce human labeling efforts. In this paper, we introduce a new framework to generate high-quality initial pixel-level annotations. By using a hierarchical image segmentation algorithm to predict the boundary map, we select the optimal scale of high-quality hierarchies. In the initialization step, scribble annotations and the saliency map are combined to construct a graphic model over the optimal scale segmentation. By solving the minimal cut problem, it can spread information from scribbles to unmarked regions. In the training process, the segmentation network is trained by using the initial pixel-level annotations. To iteratively optimize the segmentation, we use a graphical model to refine segmentation masks and retrain the segmentation network to get more precise pixel-level annotations. The experimental results on Pascal VOC 2012 dataset demonstrate that the proposed framework outperforms most of weakly supervised semantic segmentation methods and achieves the state-of-the-art performance, which is [Formula: see text] mIoU.

Download Full-text

Development of Digital Twins for Drilling Fluids: Local Velocities for Hole Cleaning and Rheology Monitoring

10.1115/omae2021-62987 ◽

2021 ◽

Author(s):

Mehrdad Gharib Shirangi ◽

Roger Aragall ◽

Reza Ettehadi ◽

Roland May ◽

Edward Furlong ◽

...

Keyword(s):

Machine Learning ◽

Rheological Properties ◽

Drilling Fluid ◽

Training Data ◽

Drilling Fluids ◽

Hole Cleaning ◽

Digital Twins ◽

Wide Range ◽

Drilling Operations ◽

Cuttings Bed

Abstract In this work, we present our advances to develop and apply digital twins for drilling fluids and associated wellbore phenomena during drilling operations. A drilling fluid digital twin is a series of interconnected models that incorporate the learning from the past historical data in a wide range of operational settings to determine the fluids properties in realtime operations. From several drilling fluid functionalities and operational parameters, we describe advancements to improve hole cleaning predictions and high-pressure high-temperature (HPHT) rheological properties monitoring. In the hole cleaning application, we consider the Clark and Bickham (1994) approach which requires the prediction of the local fluid velocity above the cuttings bed as a function of operating conditions. We develop accurate computational fluid dynamics (CFD) models to capture the effects of rotation, eccentricity and bed height on local fluid velocities above cuttings bed. We then run 55,000 CFD simulations for a wide range of operational settings to generate training data for machine learning. For rheology monitoring, thousands of lab experiment records are collected as training data for machine learning. In this case, the HPHT rheological properties are determined based on rheological measurement in the American Petroleum Institute (API) condition together with the fluid type and composition data. We compare the results of application of several machine learning algorithms to represent CFD simulations (for hole cleaning application) and lab experiments (for monitoring HPHT rheological properties). Rotating cross-validation method is applied to ensure accurate and robust results. In both cases, models from the Gradient Boosting and the Artificial Neural Network algorithms provided the highest accuracy (about 0.95 in terms of R-squared) for test datasets. With developments presented in this paper, the hole cleaning calculations can be performed more accurately in real-time, and the HPHT rheological properties of drilling fluids can be estimated at the rigsite before performing the lab experiments. These contributions advance digital transformation of drilling operations.

Download Full-text