scholarly journals Pedestrian segmentation based on a spatio-temporally consistent graph-cut with optimal transport

2019 ◽  
Vol 11 (1) ◽  
Author(s):  
Yang Yu ◽  
Yasushi Makihara ◽  
Yasushi Yagi
Keyword(s):  
Optimal Transport ◽  
Graph Cut ◽  
Temporal Consistency ◽  
Semantic Level ◽  
Spatial Consistency ◽  
Segmentation Accuracy ◽  
Spatio Temporal ◽  
Accuracy Indices ◽  
Graph Cut Segmentation ◽  
Data Term

AbstractWe address a method of pedestrian segmentation in a video in a spatio-temporally consistent way. For this purpose, given a bounding box sequence of each pedestrian obtained by a conventional pedestrian detector and tracker, we construct a spatio-temporal graph on a video and segment each pedestrian on the basis of a well-established graph-cut segmentation framework. More specifically, we consider three terms as an energy function for the graph-cut segmentation: (1) a data term, (2) a spatial pairwise term, and (3) a temporal pairwise term. To maintain better temporal consistency of segmentation even under relatively large motions, we introduce a transportation minimization framework that provides a temporal correspondence. Moreover, we introduce the edge-sticky superpixel to maintain the spatial consistency of object boundaries. In experiments, we demonstrate that the proposed method improves segmentation accuracy indices, such as the average and weighted intersection of union on TUD datasets and the PETS2009 dataset at both the instance level and semantic level.

scholarly journals Investigating the Relevance of Graph Cut Parameter on Interactive and Automatic Cell Segmentation

2018 ◽  
Vol 2018 ◽  
pp. 1-10 ◽  
Author(s):  
Kazeem Oyeyemi Oyebode ◽  
Shengzhi Du ◽  
Barend Jacobus van Wyk ◽  
Karim Djouani
Keyword(s):  
Statistical Analysis ◽  
Significant Role ◽  
Energy Function ◽  
Automatic Segmentation ◽  
Graph Cut ◽  
Cell Segmentation ◽  
Medical Field ◽  
Segmentation Strategy ◽  
Segmentation Accuracy ◽  
Graph Cut Segmentation

Graph cut segmentation provides a platform to analyze images through a global segmentation strategy, and as a result of this, it has gained a wider acceptability in many interactive and automatic segmentation fields of application, such as the medical field. The graph cut energy function has a parameter that is tuned to ensure that the output is neither oversegmented (shrink bias) nor undersegmented. Models have been proposed in literature towards the improvement of graph cut segmentation, in the context of interactive and automatic cell segmentation. Along this line of research, the graph cut parameter has been leveraged, while in some instances, it has been ignored. Therefore, in this work, the relevance of graph cut parameter on both interactive and automatic cell segmentation is investigated. Statistical analysis, based on F1 score, of three publicly available datasets of cells, suggests that the graph cut parameter plays a significant role in improving the segmentation accuracy of the interactive graph cut than the automatic graph cut.

scholarly journals Wildlife Monitoring Using a Multi-UAV System with Optimal Transport Theory

2021 ◽  
Vol 11 (9) ◽  
pp. 4070
Author(s):  
Rabiul Hasan Kabir ◽  
Kooktae Lee
Keyword(s):  
Detection Rate ◽  
Transport Theory ◽  
Optimal Transport ◽  
Position Estimation ◽  
Wildlife Monitoring ◽  
High Detection Rate ◽  
Monitoring Strategies ◽  
Spatio Temporal ◽  
Optimal Transport Theory ◽  
Multi Uav

This paper addresses a wildlife monitoring problem using a team of unmanned aerial vehicles (UAVs) with the optimal transport theory. The state-of-the-art technology using UAVs has been an increasingly popular tool to monitor wildlife compared to the traditional methods such as satellite imagery-based sensing or GPS trackers. However, there still exist unsolved problems as to how the UAVs need to cover a spacious domain to detect animals as many as possible. In this paper, we propose the optimal transport-based wildlife monitoring strategy for a multi-UAV system, to prioritize monitoring areas while incorporating complementary information such as GPS trackers and satellite-based sensing. Through the proposed scheme, the UAVs can explore the large-size domain effectively and collaboratively with a given priority. The time-varying nature of wildlife due to their movements is modeled as a stochastic process, which is included in the proposed work to reflect the spatio-temporal evolution of their position estimation. In this way, the proposed monitoring plan can lead to wildlife monitoring with a high detection rate. Various simulation results including statistical data are provided to validate the proposed work. In all different simulations, it is shown that the proposed scheme significantly outperforms other UAV-based wildlife monitoring strategies in terms of the target detection rate up to 3.6 times.

scholarly journals Glimpse: A Gaze-Based Measure of Temporal Salience

Sensors ◽  
2021 ◽  
Vol 21 (9) ◽  
pp. 3099
Author(s):  
V. Javier Traver ◽  
Judith Zorío ◽  
Luis A. Leiva
Keyword(s):  
Visual Attention ◽  
Computational Models ◽  
Temporal Evolution ◽  
Temporal Consistency ◽  
Visual Salience ◽  
Temporal Dimension ◽  
Spatial Perspective ◽  
Spatio Temporal ◽  
Scoring Algorithms ◽  
Over Time

Temporal salience considers how visual attention varies over time. Although visual salience has been widely studied from a spatial perspective, its temporal dimension has been mostly ignored, despite arguably being of utmost importance to understand the temporal evolution of attention on dynamic contents. To address this gap, we proposed Glimpse, a novel measure to compute temporal salience based on the observer-spatio-temporal consistency of raw gaze data. The measure is conceptually simple, training free, and provides a semantically meaningful quantification of visual attention over time. As an extension, we explored scoring algorithms to estimate temporal salience from spatial salience maps predicted with existing computational models. However, these approaches generally fall short when compared with our proposed gaze-based measure. Glimpse could serve as the basis for several downstream tasks such as segmentation or summarization of videos. Glimpse’s software and data are publicly available.

scholarly journals Every Frame Counts: Joint Learning of Video Segmentation and Optical Flow

2020 ◽  
Vol 34 (07) ◽  
pp. 10713-10720
Author(s):  
Mingyu Ding ◽  
Zhe Wang ◽  
Bolei Zhou ◽  
Jianping Shi ◽  
Zhiwu Lu ◽  
...  
Keyword(s):  
Optical Flow ◽  
Video Segmentation ◽  
Video Clip ◽  
Semantic Segmentation ◽  
Temporal Consistency ◽  
Flow Estimation ◽  
Optical Flow Estimation ◽  
Optical Flows ◽  
Benchmark Datasets ◽  
Spatio Temporal

A major challenge for video semantic segmentation is the lack of labeled data. In most benchmark datasets, only one frame of a video clip is annotated, which makes most supervised methods fail to utilize information from the rest of the frames. To exploit the spatio-temporal information in videos, many previous works use pre-computed optical flows, which encode the temporal consistency to improve the video segmentation. However, the video segmentation and optical flow estimation are still considered as two separate tasks. In this paper, we propose a novel framework for joint video semantic segmentation and optical flow estimation. Semantic segmentation brings semantic information to handle occlusion for more robust optical flow estimation, while the non-occluded optical flow provides accurate pixel-level temporal correspondences to guarantee the temporal consistency of the segmentation. Moreover, our framework is able to utilize both labeled and unlabeled frames in the video through joint training, while no additional calculation is required in inference. Extensive experiments show that the proposed model makes the video semantic segmentation and optical flow estimation benefit from each other and outperforms existing methods under the same settings in both tasks.

Spatio-Temporal Consistency for Semi-supervised Learning Using 3D Radar Cubes

2021 ◽  
Author(s):  
Wei-Yu Lee ◽  
Martin Dimitrievski ◽  
Ljubomir Jovanov ◽  
Wilfried Philips
Keyword(s):  
Supervised Learning ◽  
Temporal Consistency ◽  
Spatio Temporal

scholarly journals Early validation of PROBA-V GEOV1 LAI, FAPAR and FCOVER products for the continuity of the Copernicus Global Land Service

2015 ◽  
Vol XL-7/W3 ◽  
pp. 93-100 ◽  
Author(s):  
J. Sánchez ◽  
F. Camacho ◽  
R. Lacaze ◽  
B. Smets
Keyword(s):  
Ground Truth ◽  
Scientific Quality ◽  
Temporal Consistency ◽  
Area Index ◽  
Ground Truth Data ◽  
Spatial Consistency ◽  
High Resolution Imagery ◽  
Global Land ◽  
Early Validation ◽  
Using Data

This study investigates the scientific quality of the GEOV1 Leaf Area Index (LAI), Fraction of Absorbed Photosynthetically Active Radiation (FAPAR) and Fraction of Vegetation Cover (FCover) products based on PROBA-V observations. The procedure follows, as much as possible, the guidelines, protocols and metrics defined by the Land Product Validation (LPV) group of the Committee on Earth Observation Satellite (CEOS) for the validation of satellite-derived land products. This study is focused on the consistency of SPOT/VGT and PROBA-V GEOV1 products developed in the framework of the Copernicus Global Land Services, providing an early validation of PROBA-V GEOV1 products using data from November 2013 to May 2014, during the overlap period (November 2013-May 2014). The first natural year of PROBA-V GEOV1 products (2014) was considered for the rest of the quality assessment including comparisons with MODIS C5. Several criteria of performance were evaluated including product completeness, spatial consistency, temporal consistency, intra-annual precision and accuracy. Firstly, and inter-comparison with both spatial and temporal consistency were evaluated with reference satellite products (SPOT/VGT GEOV1 and MODIS C5) are presented over a network of sites (BELMANIP2.1). Secondly, the accuracy of PROBA-V GEOV1 products was evaluated against a number of concomitant agricultural sites is presented. The ground data was collected and up-scaled using high resolution imagery in the context of the FP7 ImagineS project in support of the evolution of Copernicus Land Service. Our results demonstrate that GEOV1 PROBA-V products were found spatially and temporally consistent with similar products (SPOT/VGT, MODISC5), and good agreement with limited ground truth data with an accuracy (RMSE) of 0.52 for LAI, 0.11 for FAPAR and 0.14 for FCover, showing a slight bias for FCover for higher values.

Mapping Spatio-Temporal Variability in NAO Rainfall Signatures

2021 ◽  
Author(s):  
Harry West ◽  
Nevil Quinn ◽  
Michael Horswell
Keyword(s):  
Spatial Pattern ◽  
North Atlantic ◽  
Temporal Variability ◽  
Reliable Estimate ◽  
Jet Stream ◽  
Temporal Consistency ◽  
North West ◽  
The North ◽  
Cold Spots ◽  
Spatio Temporal

<p>The North Atlantic Oscillation (NAO) is often cited as the primary atmospheric-oceanic circulation or teleconnection influencing regional climate in Great Britain. As our ability to predict the NAO several months in advance improves, it is important that we also continue to develop our spatial and temporal understanding of the rainfall signatures which the circulation produces.</p><p>We present a novel application of spatial statistics to explore variability in monthly NAO rainfall signatures using a 5km gridded monthly Standardised Precipitation Index (SPI) dataset. We first use the Getis-Ord Gi* statistic to map spatially significant hot and cold spots (clusters of high/wet and low/dry SPI values) in average monthly rainfall signatures under NAO Positive and Negative conditions over the period 1900-2015. We then look across the record and explore the temporal variability in these signatures, in other words how often a location is in a significant spatial hot/cold spot (high/low SPI) at a monthly scale under NAO Positive/Negative conditions.</p><p>The two phases of the NAO are typically more distinctive in the winter months, with stronger and more variable NAO Index values. The average monthly SPI analysis reveals a north-west/south-east ‘spatial divide’ in rainfall response. NAO Positive phases result in a southerly North Atlantic Jet Stream bringing warm and wet conditions from the tropics, increasing rainfall particularly in the north-western regions. However, under NAO Negative phases which result in a northerly Jet Stream, much drier conditions in the north-west prevail. Meanwhile in the south-eastern regions under both NAO phases a weaker and opposite wet/dry signal is observed. This north-west/south-east ‘spatial divide’ is marked by the location of spatially extensive hot/cold spots. The Getis-Ord Gi* result identifies that the spatial pattern we detect in average winter rainfall is statistically significant. Looking across the record, this NW/SE opposing response appears to have a relatively high degree of spatio-temporal consistency. This suggests that there is a high probability that NAO Positive and Negative phases will result in this NW/SE statistically significant spatial pattern.</p><p>Even though the phases of the NAO in the summer months are less distinctive they still produce rainfall responses which are evident in the monthly average SPI. However, the spatiality in wet/dry conditions is more homogenous across the country. In other words the ‘spatial divide’ observed in winter is diluted in summer. As a result, the occurrence of significant hot/cold spots is more variable in space and time.</p><p>Our analysis demonstrates a novel application of the Getis-Ord Gi* statistic which allows for spatially significant patterns in the monthly SPI data to be mapped for each NAO phase. In winter months particularly, this analysis reveals statistically significant opposing rainfall responses, which appear to have long-term spatio-temporal consistency. This is important because as winter NAO forecasting skill improves, the findings of our research enable a more spatially reliable estimate of the likely impacts of NAO-influenced rainfall distribution.</p>

Mobility Profiling

2014 ◽  
pp. 1-29
Author(s):  
Mirco Nanni ◽  
Roberto Trasarti ◽  
Paolo Cintia ◽  
Barbara Furletti ◽  
Chiara Renso ◽  
...  
Keyword(s):  
Moving Objects ◽  
Human Mobility ◽  
Large Datasets ◽  
Accurate Analysis ◽  
Interesting Phenomenon ◽  
Transportation Management ◽  
Traffic Demand ◽  
Semantic Level ◽  
Mobility Data ◽  
Spatio Temporal

The ability to understand the dynamics of human mobility is crucial for tasks like urban planning and transportation management. The recent rapidly growing availability of large spatio-temporal datasets gives us the possibility to develop sophisticated and accurate analysis methods and algorithms that can enable us to explore several relevant mobility phenomena: the distinct access paths to a territory, the groups of persons that move together in space and time, the regions of a territory that contains a high density of traffic demand, etc. All these paradigmatic perspectives focus on a collective view of the mobility where the interesting phenomenon is the result of the contribution of several moving objects. In this chapter, the authors explore a different approach to the topic and focus on the analysis and understanding of relevant individual mobility habits in order to assign a profile to an individual on the basis of his/her mobility. This process adds a semantic level to the raw mobility data, enabling further analyses that require a deeper understanding of the data itself. The studies described in this chapter are based on two large datasets of spatio-temporal data, originated, respectively, from GPS-equipped devices and from a mobile phone network.

Sign in / Sign up

Export Citation Format

Share Document