scholarly journals Spatial-Temporal Person Re-Identification

2019 ◽  
Vol 33 ◽  
pp. 8933-8940 ◽  
Author(s):  
Guangcong Wang ◽  
Jianhuang Lai ◽  
Peigen Huang ◽  
Xiaohua Xie
Keyword(s):  
Probability Distribution ◽  
Semantic Information ◽  
Temporal Information ◽  
Unified Framework ◽  
Temporal Constraint ◽  
Query Image ◽  
Similarity Metric ◽  
Heterogeneous Information ◽  
Large Margin ◽  
Previous State

Most of current person re-identification (ReID) methods neglect a spatial-temporal constraint. Given a query image, conventional methods compute the feature distances between the query image and all the gallery images and return a similarity ranked table. When the gallery database is very large in practice, these approaches fail to obtain a good performance due to appearance ambiguity across different camera views. In this paper, we propose a novel two-stream spatial-temporal person ReID (st-ReID) framework that mines both visual semantic information and spatial-temporal information. To this end, a joint similarity metric with Logistic Smoothing (LS) is introduced to integrate two kinds of heterogeneous information into a unified framework. To approximate a complex spatial-temporal probability distribution, we develop a fast Histogram-Parzen (HP) method. With the help of the spatial-temporal constraint, the st-ReID model eliminates lots of irrelevant images and thus narrows the gallery database. Without bells and whistles, our st-ReID method achieves rank-1 accuracy of 98.1% on Market-1501 and 94.4% on DukeMTMC-reID, improving from the baselines 91.2% and 83.8%, respectively, outperforming all previous state-of-theart methods by a large margin.

Multi-Resolution Autoregressive Graph-to-Graph Translation for Molecules

2019 ◽  
Author(s):  
Wengong Jin ◽  
Regina Barzilay ◽  
Tommi S Jaakkola
Keyword(s):  
Drug Discovery ◽  
State Of The Art ◽  
Molecular Graph ◽  
Biochemical Properties ◽  
Large Margin ◽  
Previous State ◽  
Translation Methods ◽  
Atom Level ◽  
Precursor Molecules ◽  
Prior State

The problem of accelerating drug discovery relies heavily on automatic tools to optimize precursor molecules to afford them with better biochemical properties. Our work in this paper substantially extends prior state-of-the-art on graph-to-graph translation methods for molecular optimization. In particular, we realize coherent multi-resolution representations by interweaving trees over substructures with the atom-level encoding of the original molecular graph. Moreover, our graph decoder is fully autoregressive, and interleaves each step of adding a new substructure with the process of resolving its connectivity to the emerging molecule. We evaluate our model on multiple molecular optimization tasks and show that our model outperforms previous state-of-the-art baselines by a large margin.

scholarly journals Using spatial-temporal ensembles of convolutional neural networks for lumen segmentation in ureteroscopy

2021 ◽  
Author(s):  
Jorge F. Lazo ◽  
Aldo Marzullo ◽  
Sara Moccia ◽  
Michele Catellani ◽  
Benoit Rosa ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
State Of The Art ◽  
Automatic Segmentation ◽  
Temporal Information ◽  
Invasive Technique ◽  
Dice Similarity Coefficient ◽  
Specular Reflections ◽  
Lumen Segmentation ◽  
Previous State

Abstract Purpose Ureteroscopy is an efficient endoscopic minimally invasive technique for the diagnosis and treatment of upper tract urothelial carcinoma. During ureteroscopy, the automatic segmentation of the hollow lumen is of primary importance, since it indicates the path that the endoscope should follow. In order to obtain an accurate segmentation of the hollow lumen, this paper presents an automatic method based on convolutional neural networks (CNNs). Methods The proposed method is based on an ensemble of 4 parallel CNNs to simultaneously process single and multi-frame information. Of these, two architectures are taken as core-models, namely U-Net based in residual blocks ($$m_1$$ m 1 ) and Mask-RCNN ($$m_2$$ m 2 ), which are fed with single still-frames I(t). The other two models ($$M_1$$ M 1 , $$M_2$$ M 2 ) are modifications of the former ones consisting on the addition of a stage which makes use of 3D convolutions to process temporal information. $$M_1$$ M 1 , $$M_2$$ M 2 are fed with triplets of frames ($$I(t-1)$$ I ( t - 1 ) , I(t), $$I(t+1)$$ I ( t + 1 ) ) to produce the segmentation for I(t). Results The proposed method was evaluated using a custom dataset of 11 videos (2673 frames) which were collected and manually annotated from 6 patients. We obtain a Dice similarity coefficient of 0.80, outperforming previous state-of-the-art methods. Conclusion The obtained results show that spatial-temporal information can be effectively exploited by the ensemble model to improve hollow lumen segmentation in ureteroscopic images. The method is effective also in the presence of poor visibility, occasional bleeding, or specular reflections.

scholarly journals A temporal constraint structure for extracting temporal information from clinical narrative

2006 ◽  
Vol 39 (4) ◽  
pp. 424-439 ◽  
Author(s):  
Li Zhou ◽  
Genevieve B. Melton ◽  
Simon Parsons ◽  
George Hripcsak
Keyword(s):  
Temporal Information ◽  
Temporal Constraint ◽  
Clinical Narrative ◽  
Constraint Structure

scholarly journals A Tri-Attention Neural Network Model-BasedRecommendation

Complexity ◽  
2020 ◽  
Vol 2020 ◽  
pp. 1-10 ◽  
Author(s):  
Nanxin Wang ◽  
Libin Yang ◽  
Yu Zheng ◽  
Xiaoyan Cai ◽  
Xin Mei ◽  
...  
Keyword(s):  
Neural Network ◽  
Semantic Information ◽  
Mutual Influence ◽  
Local Information ◽  
Information Network ◽  
Heterogeneous Information ◽  
Meta Path ◽  
Global And Local ◽  
Local Representations ◽  
Better Than

Heterogeneous information network (HIN), which contains various types of nodes and links, has been applied in recommender systems. Although HIN-based recommendation approaches perform better than the traditional recommendation approaches, they still have the following problems: for example, meta-paths are manually selected, not automatically; meta-path representations are rarely explicitly learned; and the global and local information of each node in HIN has not been simultaneously explored. To solve the above deficiencies, we propose a tri-attention neural network (TANN) model for recommendation task. The proposed TANN model applies the stud genetic algorithm to automatically select meta-paths at first. Then, it learns global and local representations of each node, as well as the representations of meta-paths existing in HIN. After that, a tri-attention mechanism is proposed to enhance the mutual influence among users, items, and their related meta-paths. Finally, the encoded interaction information among the user, the item, and their related meta-paths, which contain more semantic information can be used for recommendation task. Extensive experiments on the Douban Movie, MovieLens, and Yelp datasets have demonstrated the outstanding performance of the proposed approach.

scholarly journals Scene Text Detection with Supervised Pyramid Context Network

2019 ◽  
Vol 33 ◽  
pp. 9038-9045 ◽  
Author(s):  
Enze Xie ◽  
Yuhang Zang ◽  
Shuai Shao ◽  
Gang Yu ◽  
Cong Yao ◽  
...  
Keyword(s):  
Semantic Information ◽  
State Of The Art ◽  
Text Detection ◽  
Detection Methods ◽  
Natural Scenes ◽  
The Past ◽  
Scene Text Detection ◽  
Scene Text ◽  
Previous State ◽  
Feature Pyramid

Scene text detection methods based on deep learning have achieved remarkable results over the past years. However, due to the high diversity and complexity of natural scenes, previous state-of-the-art text detection methods may still produce a considerable amount of false positives, when applied to images captured in real-world environments. To tackle this issue, mainly inspired by Mask R-CNN, we propose in this paper an effective model for scene text detection, which is based on Feature Pyramid Network (FPN) and instance segmentation. We propose a supervised pyramid context network (SPCNET) to precisely locate text regions while suppressing false positives.Benefited from the guidance of semantic information and sharing FPN, SPCNET obtains significantly enhanced performance while introducing marginal extra computation. Experiments on standard datasets demonstrate that our SPCNET clearly outperforms start-of-the-art methods. Specifically, it achieves an F-measure of 92.1% on ICDAR2013, 87.2% on ICDAR2015, 74.1% on ICDAR2017 MLT and 82.9% on

Large margin clustering on uncertain data by considering probability distribution similarity

2015 ◽  
Vol 158 ◽  
pp. 81-89 ◽  
Author(s):  
Lei Xu ◽  
Qinghua Hu ◽  
Edward Hung ◽  
Baowen Chen ◽  
Xu Tan ◽  
...  
Keyword(s):  
Probability Distribution ◽  
Uncertain Data ◽  
Large Margin

scholarly journals Multiple Partitions Aligned Clustering

2019 ◽  
Author(s):  
Zhao Kang ◽  
Zipeng Guo ◽  
Shudong Huang ◽  
Siying Wang ◽  
Wenyu Chen ◽  
...  
Keyword(s):  
State Of The Art ◽  
Cluster Structure ◽  
Consensus Clustering ◽  
Consensus Cluster ◽  
Clustering Methods ◽  
Unified Framework ◽  
Significant Information ◽  
Heterogeneous Information ◽  
Data Points ◽  
Indicator Matrix

Multi-view clustering is an important yet challenging task due to the difficulty of integrating the information from multiple representations. Most existing multi-view clustering methods explore the heterogeneous information in the space where the data points lie. Such common practice may cause significant information loss because of unavoidable noise or inconsistency among views. Since different views admit the same cluster structure, the natural space should be all partitions. Orthogonal to existing techniques, in this paper, we propose to leverage the multi-view information by fusing partitions. Specifically, we align each partition to form a consensus cluster indicator matrix through a distinct rotation matrix. Moreover, a weight is assigned for each view to account for the clustering capacity differences of views. Finally, the basic partitions, weights, and consensus clustering are jointly learned in a unified framework. We demonstrate the effectiveness of our approach on several real datasets, where significant improvement is found over other state-of-the-art multi-view clustering methods.

MRN: Moment Relation Network for Natural Language Video Localization with Transfer Learning

2021 ◽  
pp. 2152009
Author(s):  
Siyu Jiang ◽  
Guobin Wu
Keyword(s):  
Probability Distribution ◽  
Natural Language ◽  
Transfer Learning ◽  
State Of The Art ◽  
Description Language ◽  
Large Margin ◽  
Shared Space ◽  
Learning Techniques ◽  
Benchmark Datasets ◽  
Temporal Segment

In this paper, we tackle the task of natural language video localization (NLVL): given an untrimmed video and a description language query, the goal is to localize the temporal segment within the video that best describes the natural language description. NLVL is challenging at the intersection of language and video understanding because a video may contain multiple segments of interests and the language may describe complicated temporal dependencies. Though existing approaches have achieved good performance, most of them did not fully consider the inherent differences between language and video modalities. Here, we propose Moment Relation Network (MRN) to reduce the divergence of the probability distribution of these two modalities. Specifically, MRN trains video and language subnets, and then uses transfer learning techniques to map the extracted features into an embedding-shared space where we calculate the similarity of two modalities using Mahalanobis distance metric, which is used to localize moments. Extensive experiments on benchmark datasets show that the proposed MRN significantly outperforms the state-of-the-art under the widely used metrics by a large margin.

scholarly journals Modality-Dependent Cross-Modal Retrieval Based on Graph Regularization

2020 ◽  
Vol 2020 ◽  
pp. 1-17
Author(s):  
Guanhua Wang ◽  
Hua Ji ◽  
Dexin Kong ◽  
Na Zhang
Keyword(s):  
Semantic Information ◽  
Feature Space ◽  
Heterogeneous Data ◽  
Query Image ◽  
Graph Regularization ◽  
Semantic Correlation ◽  
Image Query ◽  
Class Labels ◽  
Text Query ◽  
Original Feature

Nowadays, the heterogeneity gap of different modalities is the key problem for cross-modal retrieval. In order to overcome heterogeneity gaps, potential correlations of different modalities need to be mined. At the same time, the semantic information of class labels is used to reduce the semantic gaps between different modalities data and realize the interdependence and interoperability of heterogeneous data. In order to fully exploit the potential correlation of different modalities, we propose a cross-modal retrieval framework based on graph regularization and modality dependence (GRMD). Firstly, considering the potential feature correlation and semantic correlation, different projection matrices are learned for different retrieval tasks, such as image query text (I2T) or text query image (T2I). Secondly, utilizing the internal structure of original feature space constructs an adjacent graph with semantic information constraints which can make different labels of heterogeneous data closer to the corresponding semantic information. The experimental results on three widely used datasets demonstrate the effectiveness of our method.

Sign in / Sign up

Export Citation Format

Share Document