Superpixel-Based Attention Graph Neural Network for Semantic Segmentation in Aerial Images

Semantic segmentation is one of the significant tasks in understanding aerial images with high spatial resolution. Recently, Graph Neural Network (GNN) and attention mechanism have achieved excellent performance in semantic segmentation tasks in general images and been applied to aerial images. In this paper, we propose a novel Superpixel-based Attention Graph Neural Network (SAGNN) for semantic segmentation of high spatial resolution aerial images. A K-Nearest Neighbor (KNN) graph is constructed from our network for each image, where each node corresponds to a superpixel in the image and is associated with a hidden representation vector. On this basis, the initialization of the hidden representation vector is the appearance feature extracted by a unary Convolutional Neural Network (CNN) from the image. Moreover, relying on the attention mechanism and recursive functions, each node can update its hidden representation according to the current state and the incoming information from its neighbors. The final representation of each node is used to predict the semantic class of each superpixel. The attention mechanism enables graph nodes to differentially aggregate neighbor information, which can extract higher-quality features. Furthermore, the superpixels not only save computational resources, but also maintain object boundary to achieve more accurate predictions. The accuracy of our model on the Potsdam and Vaihingen public datasets exceeds all benchmark approaches, reaching 90.23% and 89.32%, respectively.

An efficient and fast multi-band focused bioimpedance solution with EIT-based reconstruction for pulmonary embolism assessment: a simulation study from massive to segmental blockage

Objective: Pulmonary embolism (PE) is an acute condition that blocks the perfusion to the lungs and is a common complication of Covid-19. However, PE is often not diagnosed in time, especially in the pandemic time due to complicated diagnosis protocol. In this study, a non-invasive, fast and efficient bioimpedance method with the EIT-based reconstruction approach is proposed to assess the lung perfusion reliably. Approach: Some proposals are presented to improve the sensitivity and accuracy for the bioimpedance method: (1) a new electrode configuration and focused pattern to help study deep changes caused by PE within each lung field separately, (2) a measurement strategy to compensate the effect of different boundary shapes and varied respiratory conditions on the perfusion signals and (3) an estimator to predict the lung perfusion capacity, from which the severity of PE can be assessed. The proposals were tested on the first-time simulation of PE events at different locations and degrees from segmental blockages to massive blockages. Different object boundary shapes and varied respiratory conditions were included in the simulation to represent for different populations in real measurements. Results: The correlation between the estimator and the perfusion was very promising (R = 0.91, errors < 6%). The measurement strategy with the proposed configuration and pattern has helped stabilize the estimator to non-perfusion factors such as the boundary shapes and varied respiration conditions (3-5% errors). Significance: This promising preliminary result has demonstrated the proposed bioimpedance method’s capability and feasibility, and might start a new direction for this application.

Effective connectivity of LSD-induced ego dissolution

Classic psychedelic-induced ego dissolution involves a shift in the sense of self and blurring of boundary between the self and the world. A similar phenomenon is identified in psychopathology and is associated to the balance of anticorrelated activity between the default mode network (DMN) – which directs attention inwards – and the salience network (SN) – which recruits the dorsal attention network (DAN) to direct attention outward. To test whether change in anticorrelated networks underlie the peak effects of LSD, we applied dynamic causal modeling to infer effective connectivity of resting state functional MRI scans from a study of 25 healthy adults who were administered 100mg of LSD, or placebo. We found that change in inhibitory effective connectivity from the SN to DMN became excitatory, and inhibitory effective connectivity from DMN to DAN decreased under the peak effect of LSD. These changes in connectivity reflect diminution of the anticorrelation between resting state networks that may be a key neural mechanism of LSD-induced ego dissolution. Our findings suggest the hierarchically organised balance of resting state networks is a central feature in the construct of self.SignificanceThe findings can inform the parallel between the maintenance of subject-object boundary and changes to anticorrelated canonical resting state brain networks. Effective connectivity informs the hierarchical organisation of brain networks underlying modes of perception. Moreover, the anticorrelation of brain networks is an important measure of mental function. Understanding the neural mechanisms of anticorrelation change under psychedelics help identify its relationship to psychosis and its association to psychedelic assisted therapeutic outcomes.

Towards natural object-based image recoloring

Existing color editing algorithms enable users to edit the colors in an image according to their own aesthetics. Unlike artists who have an accurate grasp of color, ordinary users are inexperienced in color selection and matching, and allowing non-professional users to edit colors arbitrarily may lead to unrealistic editing results. To address this issue, we introduce a palette-based approach for realistic object-level image recoloring. Our data-driven approach consists of an offline learning part that learns the color distributions for different objects in the real world, and an online recoloring part that first recognizes the object category, and then recommends appropriate realistic candidate colors learned in the offline step for that category. We also provide an intuitive user interface for efficient color manipulation. After color selection, image matting is performed to ensure smoothness of the object boundary. Comprehensive evaluation on various color editing examples demonstrates that our approach outperforms existing state-of-the-art color editing algorithms.

Dual Attention Feature Fusion and Adaptive Context for Accurate Segmentation of Very High-Resolution Remote Sensing Images

Land cover classification of high-resolution remote sensing images aims to obtain pixel-level land cover understanding, which is often modeled as semantic segmentation of remote sensing images. In recent years, convolutional network (CNN)-based land cover classification methods have achieved great advancement. However, previous methods fail to generate fine segmentation results, especially for the object boundary pixels. In order to obtain boundary-preserving predictions, we first propose to incorporate spatially adapting contextual cues. In this way, objects with similar appearance can be effectively distinguished with the extracted global contextual cues, which are very helpful to identify pixels near object boundaries. On this basis, low-level spatial details and high-level semantic cues are effectively fused with the help of our proposed dual attention mechanism. Concretely, when fusing multi-level features, we utilize the dual attention feature fusion module based on both spatial and channel attention mechanisms to relieve the influence of the large gap, and further improve the segmentation accuracy of pixels near object boundaries. Extensive experiments were carried out on the ISPRS 2D Semantic Labeling Vaihingen data and GaoFen-2 data to demonstrate the effectiveness of our proposed method. Our method achieves better performance compared with other state-of-the-art methods.

Towards improving edge quality using combinatorial optimization and a novel skeletonize algorithm

Background Object detection and image segmentation of regions of interest provide the foundation for numerous pipelines across disciplines. Robust and accurate computer vision methods are needed to properly solve image-based tasks. Multiple algorithms have been developed to solely detect edges in images. Constrained to the problem of creating a thin, one-pixel wide, edge from a predicted object boundary, we require an algorithm that removes pixels while preserving the topology. Thanks to skeletonize algorithms, an object boundary is transformed into an edge; contrasting uncertainty with exact positions. Methods To extract edges from boundaries generated from different algorithms, we present a computational pipeline that relies on: a novel skeletonize algorithm, a non-exhaustive discrete parameter search to find the optimal parameter combination of a specific post-processing pipeline, and an extensive evaluation using three data sets from the medical and natural image domains (kidney boundaries, NYU-Depth V2, BSDS 500). While the skeletonize algorithm was compared to classical topological skeletons, the validity of our post-processing algorithm was evaluated by integrating the original post-processing methods from six different works. Results Using the state of the art metrics, precision and recall based Signed Distance Error (SDE) and the Intersection over Union bounding box (IOU-box), our results indicate that the SDE metric for these edges is improved up to 2.3 times. Conclusions Our work provides guidance for parameter tuning and algorithm selection in the post-processing of predicted object boundaries.

Comparative studies of multiscale edge detection using different edge detectors for MRI thigh

Edge detection plays an important role in computer vision to extract object boundary. Multiscale edge detection method provides a variety of image features by different resolution at multiscale of edges. The method extracts coarse and fine structure edges simultaneously in an image. Due to this, the multiscale method enables more reliable edges are detected. Most of the multiscale methods are not translation invariant due to the decimated process. They mostly depend on the corresponding transform coefficients. These methods need more computation and a larger storage space. This study proposes a multiscale method that uses an average filter to smooth image at three different scales. Three different classical edge detectors namely Prewitt, Sobel and Laplacian were used to extract the edges from the smooth images. The edges extracted from the different scales of smooth images were then combined to form the multiscale edge detection. The performances of the multiscale images extracted from the three classical edge detectors were then compared and discussed.

Lychee Surface Defect Detection Based on Deep Convolutional Neural Networks with GAN-Based Data Augmentation

The performance of fruit surface defect detection is easily affected by factors such as noisy background and foliage occlusion. In this study, we choose lychee as a fruit type to investigate its surface quality. Lychees are hard to preserve and have to be stored at low temperatures to keep fresh. Additionally, the surface of lychees is subject to scratches and cracks during harvesting/processing. To explore the feasibility of the automation of defective surface detection for lychees, we build a dataset with 3743 samples divided into three categories, namely, mature, defects, and rot. The original dataset suffers an imbalanced distribution issue. To address it, we adopt a transformer-based generative adversarial network (GAN) as a means of data augmentation that can effectively enhance the original training set with more and diverse samples to rebalance the three categories. In addition, we investigate three deep convolutional neural network (DCNN) models, including SSD-MobileNet V2, Faster RCNN-ResNet50, and Faster RCNN-Inception-ResNet V2, trained under different settings for an extensive comparison study. The results show that all three models demonstrate consistent performance gains in mean average precision (mAP), with the application of GAN-based augmentation. The rebalanced dataset also reduces the inter-category discrepancy, allowing a DCNN model to be trained equally across categories. In addition, the qualitative results show that models trained under the augmented setting can better identify the critical regions and the object boundary, leading to gains in mAP. Lastly, we conclude that the most cost-effective model, SSD-MobileNet V2, presents a comparable mAP (91.81%) and a superior inference speed (102 FPS), suitable for real-time detection in industrial-level applications.