Joint Classification and Regression for Visual Tracking with Fully Convolutional Siamese Networks

Visual tracking of generic objects is one of the fundamental but challenging problems in computer vision. Here, we propose a novel fully convolutional Siamese network to solve visual tracking by directly predicting the target bounding box in an end-to-end manner. We first reformulate the visual tracking task as two subproblems: a classification problem for pixel category prediction and a regression task for object status estimation at this pixel. With this decomposition, we design a simple yet effective Siamese architecture based classification and regression framework, termed SiamCAR, which consists of two subnetworks: a Siamese subnetwork for feature extraction and a classification-regression subnetwork for direct bounding box prediction. Since the proposed framework is both proposal- and anchor-free, SiamCAR can avoid the tedious hyper-parameter tuning of anchors, considerably simplifying the training. To demonstrate that a much simpler tracking framework can achieve superior tracking results, we conduct extensive experiments and comparisons with state-of-the-art trackers on a few challenging benchmarks. Without bells and whistles, SiamCAR achieves leading performance with a real-time speed. Furthermore, the ablation study validates that the proposed framework is effective with various backbone networks, and can benefit from deeper networks. Code is available at https://github.com/ohhhyeahhh/SiamCAR.

High Precision Mammography Lesion Identification From Imprecise Medical Annotations

Breast cancer screening using Mammography serves as the earliest defense against breast cancer, revealing anomalous tissue years before it can be detected through physical screening. Despite the use of high resolution radiography, the presence of densely overlapping patterns challenges the consistency of human-driven diagnosis and drives interest in leveraging state-of-art localization ability of deep convolutional neural networks (DCNN). The growing availability of digitized clinical archives enables the training of deep segmentation models, but training using the most widely available form of coarse hand-drawn annotations works against learning the precise boundary of cancerous tissue in evaluation, while producing results that are more aligned with the annotations rather than the underlying lesions. The expense of collecting high quality pixel-level data in the field of medical science makes this even more difficult. To surmount this fundamental challenge, we propose LatentCADx, a deep learning segmentation model capable of precisely annotating cancer lesions underlying hand-drawn annotations, which we procedurally obtain using joint classification training and a strict segmentation penalty. We demonstrate the capability of LatentCADx on a publicly available dataset of 2,620 Mammogram case files, where LatentCADx obtains classification ROC of 0.97, AP of 0.87, and segmentation AP of 0.75 (IOU = 0.5), giving comparable or better performance than other models. Qualitative and precision evaluation of LatentCADx annotations on validation samples reveals that LatentCADx increases the specificity of segmentations beyond that of existing models trained on hand-drawn annotations, with pixel level specificity reaching a staggering value of 0.90. It also obtains sharp boundary around lesions unlike other methods, reducing the confused pixels in the output by more than 60%.

An Initial Investigation of the Joint Classification of Hypomania- and Neurodevelopmental Disorder-Relevant Dimensions Within the Hierarchical Taxonomy of Psychopathology

We examined hierarchical structural models of psychopathology in samples of (a) adults recruited online and screened based on psychopathology history ( N = 429) and (b) undergraduates ( N = 529) to inform classification of neurodevelopmental disorder (NDD)- and hypomania-relevant dimensions within the Hierarchical Taxonomy of Psychopathology (HiTOP). Results differed across samples in some ways, but converged to indicate that some NDD- and hypomania-relevant dimensions aligned closely with different HiTOP spectra. For example, some hypomania-relevant dimensions (e.g., affective lability) overlapped strongly with the internalizing spectrum, whereas others (e.g., self-perceived charisma) were reverse-indicators of detachment. Examination of cross-sectional and prospective correlates for emergent factors also was informative in some ways. This included NDD-relevant and disinhibited externalizing dimensions associating robustly with treatment seeking history and recent experiences of distress. These results provide initial insights into classifying NDD- and hypomania-relevant dimensions within the HiTOP and indicate a need for future research in this area.

Deep Multi-Segmentation Approach for the Joint Classification and Segmentation of the Retinal Arterial and Venous Trees in Color Fundus Images

The analysis of the retinal vasculature represents a crucial stage in the diagnosis of several diseases. An exhaustive analysis involves segmenting the retinal vessels and classifying them into veins and arteries. In this work, we present an accurate approach, based on deep neural networks, for the joint segmentation and classification of the retinal veins and arteries from color fundus images. The presented approach decomposes this joint task into three related subtasks: the segmentation of arteries, veins and the whole vascular tree. The experiments performed show that our method achieves competitive results in the discrimination of arteries and veins, while clearly enhancing the segmentation of the different structures. Moreover, unlike other approaches, our method allows for the straightforward detection of vessel crossings, and preserves the continuity of the arterial and venous vascular trees at these locations.

JOINT LAND COVER AND CROP TYPE MAPPING USING MULTI-TEMPORAL SENTINEL-2 DATA FROM VARIOUS ENVIRONMENTAL ZONES IN GREECE

In this work, we elaborate on the gained insights from various classification experiments towards detailed land cover mapping over four representative regions of different environmental characteristics in Greece. In particular, the proposed methodology exploits Sentinel-2 data at an annual basis, for the joint classification of 35 land cover and crop type classes. A number of pre-processing steps were employed on the satellite data, in order to address atmospheric and geometric effects, as well as clouds and pertinent shadows. Several classification set-ups were designed and performed using either time series of spectral features or temporal features. The latter consisted of statistical metrics, derived from the spectral time series, and therefore were significantly reduced in dimension. Experiments using the Random Forest algorithm were performed by building several per-tile models, as well as cross- regional models based on training data from all considered regions/tiles. Overall classification accuracy rates exceeded 90% for most experiments. Further analysis on the experimental results highlighted that crop types were classified more accurately when using the spectral time series features, compared to the temporal ones. Classification accuracy for non-crop classes proved much less affected by the type of employed features. The inclusion of auxiliary data layers was beneficial in all cases, both for overall and for per-class accuracy metrics. Qualitative evaluation on the predicted maps further affirmed the efficiency of the developed methodology.

Bimodal Data Fusion of Simultaneous Measurements of EEG and fNIRS during Lower Limb Movements

Electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) have temporal and spatial characteristics that may complement each other and, therefore, pose an intriguing approach for brain-computer interaction (BCI). In this work, the relationship between the hemodynamic response and brain oscillation activity was investigated using the concurrent recording of fNIRS and EEG during ankle joint movements. Twenty subjects participated in this experiment. The EEG was recorded using 20 electrodes and hemodynamic responses were recorded using 32 optodes positioned over the motor cortex areas. The event-related desynchronization (ERD) feature was extracted from the EEG signal in the alpha band (8–11) Hz, and the concentration change of the oxy-hemoglobin (oxyHb) was evaluated from the hemodynamics response. During the motor execution of the ankle joint movements, a decrease in the alpha (8–11) Hz amplitude (desynchronization) was found to be correlated with an increase of the oxyHb (r = −0.64061, p < 0.00001) observed on the Cz electrode and the average of the fNIRS channels (ch28, ch25, ch32, ch35) close to the foot area representation. Then, the correlated channels in both modalities were used for ankle joint movement classification. The result demonstrates that the integrated modality based on the correlated channels provides a substantial enhancement in ankle joint classification accuracy of 93.01 ± 5.60% (p < 0.01) compared with single modality. These results highlight the potential of the bimodal fNIR–EEG approach for the development of future BCI for lower limb rehabilitation.

Combination of Joint Representation and Adaptive Weighting for Multiple Features with Application to SAR Target Recognition

For the synthetic aperture radar (SAR) target recognition problem, a method combining multifeature joint classification and adaptive weighting is proposed with innovations in fusion strategies. Zernike moments, nonnegative matrix factorization (NMF), and monogenic signal are employed as the feature extraction algorithms to describe the characteristics of original SAR images with three corresponding feature vectors. Based on the joint sparse representation model, the three types of features are jointly represented. For the reconstruction error vectors from different features, an adaptive weighting algorithm is used for decision fusion. That is, the weights are adaptively obtained under the framework of linear fusion to achieve a good fusion result. Finally, the target label is determined according to the fused error vector. Experiments are conducted on the moving and stationary target acquisition and recognition (MSTAR) dataset under the standard operating condition (SOC) and four extended operating conditions (EOC), i.e., configuration variants, depression angle variances, noise interference, and partial occlusion. The results verify the effectiveness and robustness of the proposed method.