scholarly journals Learning Multi-human Optical Flow

2020 ◽  
Vol 128 (4) ◽  
pp. 873-890 ◽  
Author(s):  
Anurag Ranjan ◽  
David T. Hoffmann ◽  
Dimitrios Tzionas ◽  
Siyu Tang ◽  
Javier Romero ◽  
...  
Keyword(s):  
Optical Flow ◽  
3D Model ◽  
Flow Fields ◽  
Human Action ◽  
Human Motion ◽  
Image Sequences ◽  
Training Data ◽  
Flow Networks ◽  
Wide Range ◽  
Human Flow

AbstractThe optical flow of humans is well known to be useful for the analysis of human action. Recent optical flow methods focus on training deep networks to approach the problem. However, the training data used by them does not cover the domain of human motion. Therefore, we develop a dataset of multi-human optical flow and train optical flow networks on this dataset. We use a 3D model of the human body and motion capture data to synthesize realistic flow fields in both single- and multi-person images. We then train optical flow networks to estimate human flow fields from pairs of images. We demonstrate that our trained networks are more accurate than a wide range of top methods on held-out test data and that they can generalize well to real image sequences. The code, trained models and the dataset are available for research.

INDEPENDENT COMPONENTS OF OPTICAL FLOW IN A MULTIRESOLUTION IMAGE SEQUENCE

2008 ◽  
Vol 22 (05) ◽  
pp. 929-944 ◽  
Author(s):  
NAOYA OHNISHI ◽  
ATSUSHI IMIYA
Keyword(s):  
Mobile Robot ◽  
Optical Flow ◽  
Robot Vision ◽  
Image Sequence ◽  
Flow Fields ◽  
Image Sequences ◽  
Visual Navigation ◽  
Local Motion ◽  
Independent Components ◽  
Local Perception

In this paper, we present an algorithm for the hierarchical recognition of an environment using independent components of optical flow fields for the visual navigation of a mobile robot. For the computation of optical flow, the pyramid transform of an image sequence is used for the analysis of global and local motion. Our algorithm detects the planar region and obstacles in the image from optical flow fields at each layer in the pyramid. Therefore, our algorithm allows us to achieve both global perception and local perception for robot vision. We show experimental results for both test image sequences and real image sequences captured by a mobile robot. Furthermore, we show some aspects of this work from the viewpoint of information theory.

IMAGE SEQUENCES—TEN (OCTAL) YEARS—FROM PHENOMENOLOGY TOWARDS A THEORETICAL FOUNDATION

1988 ◽  
Vol 02 (03) ◽  
pp. 459-483 ◽  
Author(s):  
HANS-HELLMUT NAGEL
Keyword(s):  
Optical Flow ◽  
Autonomous Vehicles ◽  
Theoretical Foundation ◽  
Image Sequence ◽  
Flow Fields ◽  
Image Sensor ◽  
Image Sequences ◽  
Common Foundation ◽  
Feature Based ◽  
Spatio Temporal

Many investigations of image sequences can be understood on the basis of a few concepts for which computational approaches become increasingly available. The estimation of optical flow fields is discussed, exhibiting a common foundation for feature-based and differential approaches. The interpretation of optical flow fields is mostly concerned so far with approaches which infer the 3-D structure of a rigid point configuration in 3-D space and its relative motion with respect to the image sensor from an image sequence. The combination of stereo and motion provides additional incentives to evaluate image sequences, especially for the control of robots and autonomous vehicles. Advances in all these areas lead to the desire to describe the spatio-temporal development recorded by an image sequence not only at the level of geometry, but also at higher conceptual levels, for example by natural language descriptions.

scholarly journals A Driver’s Visual Attention Prediction Using Optical Flow

Sensors ◽  
2021 ◽  
Vol 21 (11) ◽  
pp. 3722
Author(s):  
Byeongkeun Kang ◽  
Yeejin Lee
Keyword(s):  
Visual Attention ◽  
Optical Flow ◽  
Prediction Models ◽  
Apparent Movement ◽  
Image Sequences ◽  
Motion Information ◽  
Accuracy Improvement ◽  
Flow Maps ◽  
Current State ◽  
Motion Features

Motion in videos refers to the pattern of the apparent movement of objects, surfaces, and edges over image sequences caused by the relative movement between a camera and a scene. Motion, as well as scene appearance, are essential features to estimate a driver’s visual attention allocation in computer vision. However, the fact that motion can be a crucial factor in a driver’s attention estimation has not been thoroughly studied in the literature, although driver’s attention prediction models focusing on scene appearance have been well studied. Therefore, in this work, we investigate the usefulness of motion information in estimating a driver’s visual attention. To analyze the effectiveness of motion information, we develop a deep neural network framework that provides attention locations and attention levels using optical flow maps, which represent the movements of contents in videos. We validate the performance of the proposed motion-based prediction model by comparing it to the performance of the current state-of-art prediction models using RGB frames. The experimental results for a real-world dataset confirm our hypothesis that motion plays a role in prediction accuracy improvement, and there is a margin for accuracy improvement by using motion features.

scholarly journals Unsupervised content-preserving transformation for optical microscopy

2021 ◽  
Vol 10 (1) ◽  
Author(s):  
Xinyang Li ◽  
Guoxun Zhang ◽  
Hui Qiao ◽  
Feng Bao ◽  
Yue Deng ◽  
...  
Keyword(s):  
Deep Learning ◽  
Optical Microscopy ◽  
Training Data ◽  
Fluorescence Labeling ◽  
Imaging Data ◽  
Image Transformation ◽  
General Applicability ◽  
Data Annotation ◽  
Biomedical Image ◽  
Wide Range

AbstractThe development of deep learning and open access to a substantial collection of imaging data together provide a potential solution for computational image transformation, which is gradually changing the landscape of optical imaging and biomedical research. However, current implementations of deep learning usually operate in a supervised manner, and their reliance on laborious and error-prone data annotation procedures remains a barrier to more general applicability. Here, we propose an unsupervised image transformation to facilitate the utilization of deep learning for optical microscopy, even in some cases in which supervised models cannot be applied. Through the introduction of a saliency constraint, the unsupervised model, named Unsupervised content-preserving Transformation for Optical Microscopy (UTOM), can learn the mapping between two image domains without requiring paired training data while avoiding distortions of the image content. UTOM shows promising performance in a wide range of biomedical image transformation tasks, including in silico histological staining, fluorescence image restoration, and virtual fluorescence labeling. Quantitative evaluations reveal that UTOM achieves stable and high-fidelity image transformations across different imaging conditions and modalities. We anticipate that our framework will encourage a paradigm shift in training neural networks and enable more applications of artificial intelligence in biomedical imaging.

Sign in / Sign up

Export Citation Format

Share Document