3D Vehicle Trajectory Extraction Using DCNN in an Overlapping Multi-Camera Crossroad Scene

The 3D vehicle trajectory in complex traffic conditions such as crossroads and heavy traffic is practically very useful in autonomous driving. In order to accurately extract the 3D vehicle trajectory from a perspective camera in a crossroad where the vehicle has an angular range of 360 degrees, problems such as the narrow visual angle in single-camera scene, vehicle occlusion under conditions of low camera perspective, and lack of vehicle physical information must be solved. In this paper, we propose a method for estimating the 3D bounding boxes of vehicles and extracting trajectories using a deep convolutional neural network (DCNN) in an overlapping multi-camera crossroad scene. First, traffic data were collected using overlapping multi-cameras to obtain a wide range of trajectories around the crossroad. Then, 3D bounding boxes of vehicles were estimated and tracked in each single-camera scene through DCNN models (YOLOv4, multi-branch CNN) combined with camera calibration. Using the abovementioned information, the 3D vehicle trajectory could be extracted on the ground plane of the crossroad by calculating results obtained from the overlapping multi-camera with a homography matrix. Finally, in experiments, the errors of extracted trajectories were corrected through a simple linear interpolation and regression, and the accuracy of the proposed method was verified by calculating the difference with ground-truth data. Compared with other previously reported methods, our approach is shown to be more accurate and more practical.

Download Full-text

Enhancing unsupervised video-based vehicle tracking and modeling for traffic data collection

Canadian Journal of Civil Engineering ◽

10.1139/cjce-2019-0087 ◽

2020 ◽

Vol 47 (8) ◽

pp. 982-997

Author(s):

Mohamed H. Zaki ◽

Tarek Sayed ◽

Moataz Billeh

Keyword(s):

Data Collection ◽

Video Analysis ◽

Three Dimensional ◽

Ground Truth ◽

Traffic Data ◽

Ground Truth Data ◽

The Road ◽

Traffic Data Collection ◽

Bounding Boxes ◽

Automated Video Analysis

Video-based traffic analysis is a leading technology for streamlining transportation data collection. With traffic records from video cameras, unsupervised automated video analysis can detect various vehicle measures such as vehicle spatial coordinates and subsequently lane positions, speed, and other dynamic measures without the need of any physical interconnections to the road infrastructure. This paper contributes to the unsupervised automated video analysis by addressing two main shortcomings of the approach. The first objective is to alleviate tracking problems of over-segmentation and over-grouping by integrating region-based detection with feature-based tracking. This information, when combined with spatiotemporal constraints of grouping, can reduce the effects of these problems. This fusion approach offers a superior decision procedure for grouping objects and discriminating between trajectories of objects. The second objective is to model three-dimensional bounding boxes for the vehicles, leading to a better estimate of their geometry and consequently accurate measures of their position and travel information. This improvement leads to more precise measurement of traffic parameters such as average speed, gap time, and headway. The paper describes the various steps of the proposed improvements. It evaluates the effectiveness of the refinement process on data collected from traffic cameras in three different locations in Canada and validates the results with ground truth data. It illustrates the effectiveness of the improved unsupervised automated video analysis with a case study on 10 h of traffic data collection such as volume and headway measurements.

Download Full-text

Continuous Urban Tree Cover Mapping from Landsat Imagery in Bengaluru, India

Forests ◽

10.3390/f12020220 ◽

2021 ◽

Vol 12 (2) ◽

pp. 220

Author(s):

Nils Nölke

Keyword(s):

Landsat Imagery ◽

Ground Truth ◽

Urban Environments ◽

Urban Trees ◽

Tree Cover ◽

Neural Network Approach ◽

Ground Truth Data ◽

High Resolution Imagery ◽

Wide Range ◽

Very High Resolution Imagery

Percent tree cover maps derived from Landsat imagery provide a useful data source for monitoring changes in tree cover over time. Urban trees are a special group of trees outside forests (TOFs) and occur often as solitary trees, in roadside alleys and in small groups, exhibiting a wide range of crown shapes. Framed by house walls and with impervious surfaces as background and in the immediate neighborhood, they are difficult to assess from Landsat imagery with a 30 m pixel size. In fact, global maps based on Landsat partly failed to detect a considerable portion of urban trees. This study presents a neural network approach applied to the urban trees in the metropolitan area of Bengaluru, India, resulting in a new map of estimated tree cover (MAE = 13.04%); this approach has the potential to also detect smaller trees within cities. Our model was trained with ground truth data from WorldView-3 very high resolution imagery, which allows to assess tree cover per pixel from 0% to 100%. The results of this study may be used to improve the accuracy of Landsat-based time series of tree cover in urban environments.

Download Full-text

CarFree: Hassle-Free Object Detection Dataset Generation Using Carla Autonomous Driving Simulator

Applied Sciences ◽

10.3390/app12010281 ◽

2021 ◽

Vol 12 (1) ◽

pp. 281

Author(s):

Jaesung Jang ◽

Hyeongyu Lee ◽

Jong-Chan Kim

Keyword(s):

Object Detection ◽

Real World ◽

Driving Simulator ◽

Data Extraction ◽

Ground Truth ◽

Autonomous Driving ◽

Relevant Information ◽

Detection Accuracy ◽

Daunting Task ◽

Bounding Boxes

For safe autonomous driving, deep neural network (DNN)-based perception systems play essential roles, where a vast amount of driving images should be manually collected and labeled with ground truth (GT) for training and validation purposes. After observing the manual GT generation’s high cost and unavoidable human errors, this study presents an open-source automatic GT generation tool, CarFree, based on the Carla autonomous driving simulator. By that, we aim to democratize the daunting task of (in particular) object detection dataset generation, which was only possible by big companies or institutes due to its high cost. CarFree comprises (i) a data extraction client that automatically collects relevant information from the Carla simulator’s server and (ii) a post-processing software that produces precise 2D bounding boxes of vehicles and pedestrians on the gathered driving images. Our evaluation results show that CarFree can generate a considerable amount of realistic driving images along with their GTs in a reasonable time. Moreover, using the synthesized training images with artificially made unusual weather and lighting conditions, which are difficult to obtain in real-world driving scenarios, CarFree significantly improves the object detection accuracy in the real world, particularly in the case of harsh environments. With CarFree, we expect its users to generate a variety of object detection datasets in hassle-free ways.

Download Full-text

Spatio-Temporal Variation of Vegetation in Godavari Eastern Delta

Current Journal of Applied Science and Technology ◽

10.9734/cjast/2019/v38i630412 ◽

2019 ◽

pp. 1-7

Author(s):

G. Kishore Kumar ◽

M. Raghu Babu ◽

A. Mani ◽

M. Matin Luther ◽

V. Srinivasa Rao

Keyword(s):

Vegetation Index ◽

Near Infrared ◽

Normalized Difference Vegetation Index ◽

Land Use Changes ◽

Satellite Image ◽

Ground Truth ◽

Likelihood Method ◽

Landsat Images ◽

Ground Truth Data ◽

Wide Range

Spatial variability in land use changes creates a need for a wide range of applications, including landslide, erosion, land planning, global warming etc. This study presents the analysis of satellite image based on Normalized Difference Vegetation Index (NDVI) in Godavari eastern delta. Four spectral indices were investigated in this study. These indices were NIR (red and near infrared) based NDVI, green and NIR based GVI (Green Vegetation Index), red and NIR based soil adjusted vegetation index (SAVI), and red and NIR based perpendicular vegetation index (PVI). These four indices were investigated for 2011-12 kharif, rabi and 2016-17 kharif, rabi of Godavari eastern delta. Different threshold values of NDVI are used for generating the false colour composite of the classified objects. For this purpose, supervised classification is applied to Landsat images acquired in 2011-12 and 2016-17. Image classification of six reflective bands of two Landsat images is carried out by using maximum likelihood method with the aid of ground truth data obtained from satellite images of 2011-12 and 2016-17. There was 11% and 30% increase in vegetation during kharif and rabi seasons from 2011-12 to 2016-17. The vegetation analysis can be used to provide humanitarian aid, damage assessment in case of unfortunate natural disasters and furthermore to device new protection strategies.

Download Full-text

Systematic assessment of the accuracy of subunit counting in biomolecular complexes using automated single molecule brightness analysis

10.1101/2021.11.22.469513 ◽

2021 ◽

Author(s):

John S H Danial ◽

Yuri Quintana ◽

Uris Ros ◽

Raed Shalaby ◽

Eleonora Germana Margheritis ◽

...

Keyword(s):

Single Molecule ◽

Ground Truth ◽

Experimental Conditions ◽

Systematic Analysis ◽

Ground Truth Data ◽

Systematic Assessment ◽

Brightness Analysis ◽

Wide Range ◽

Biomolecular Complexes ◽

Biological Complexes

Analysis of single molecule brightness allows subunit counting of high-order oligomeric biomolecular complexes. Although the theory behind the method has been extensively assessed, systematic analysis of the experimental conditions required to accurately quantify the stoichiometry of biological complexes remains challenging. In this work, we develop a high-throughput, automated computational pipeline for single molecule brightness analysis that requires minimal human input. We use this strategy to systematically quantify the accuracy of counting under a wide range of experimental conditions in simulated ground-truth data and then validate its use on experimentally obtained data. Our approach defines a set of conditions under which subunit counting by brightness analysis is designed to work optimally and helps establishing the experimental limits in quantifying the number of subunits in a complex of interest. Finally, we combine these features into a powerful, yet simple, software that can be easily used for the stoichiometry analysis of such complexes.

Download Full-text

On the design of a fractal calibration pattern for improved camera calibration

tm - Technisches Messen ◽

10.1515/teme-2017-0013 ◽

2017 ◽

Vol 84 (7-8) ◽

Cited By ~ 1

Author(s):

Hendrik Schilling ◽

Maximilian Diebold ◽

Marcel Gutsche ◽

Bernd Jähne

Keyword(s):

Camera Calibration ◽

Large Range ◽

Ground Truth ◽

High Accuracy ◽

Ground Truth Data ◽

Magnification Factors ◽

Wide Range ◽

Calibration Pattern ◽

Planar Calibration ◽

Very High

AbstractCamera calibration, crucial for computer vision tasks, often relies on planar calibration targets to calibrate the camera parameters. This work describes the design of a planar, fractal, self-identifying calibration pattern, which provides a high density of calibration points for a large range of magnification factors. An evaluation on ground truth data shows that the target provides very high accuracy over a wide range of conditions.

Download Full-text

Virtual Reality Providing Ground Truth for Machine Learning and Programming by Demonstration

Volume 2: 32nd Computers and Information in Engineering Conference, Parts A and B ◽

10.1115/detc2012-70375 ◽

2012 ◽

Cited By ~ 2

Author(s):

Jürgen Roßmann ◽

Christian Schlette ◽

Nils Wantia

Keyword(s):

Virtual Reality ◽

Space Station ◽

Ground Truth ◽

Cognitive System ◽

Automated System ◽

Ground Truth Data ◽

Free Data ◽

Funded Project ◽

Manual Tasks ◽

Bounding Boxes

In this contribution, it is demonstrated how the development, parameterization and evaluation of an intelligent cognitive system can be supported by means of Virtual Reality (VR). Based on new approaches in object and action recognition, the consortium in the EU funded project IntellAct is developing such an automated system for the intelligent understanding of manual tasks and robot-based manipulations. In this context, the VR setup of the Institute for Man-Machine Interaction (MMI) is able to deliver ideal data, such as detailed positions and joint data of objects and kinematics involved. In addition, it allows for the synthesis of distinct and error-free data, such as collisions, bounding boxes and contact events. This way, Virtual Reality serves as a reference and source for “ground truth” data for designing, training and benchmarking the intelligent cognitive system during its development and parameterization. Furthermore this approach enables descriptive and secure ways for the visualization of results in the targeted environments, such as the highly automated laboratories on-board the International Space Station (see Fig. 1).

Download Full-text

CLUBS: An RGB-D dataset with cluttered box scenes containing household objects

The International Journal of Robotics Research ◽

10.1177/0278364919875221 ◽

2019 ◽

Vol 38 (14) ◽

pp. 1538-1548

Author(s):

Tonci Novkovic ◽

Fadri Furrer ◽

Marko Panjek ◽

Margarita Grinvald ◽

Roland Siegwart ◽

...

Keyword(s):

Ground Truth ◽

Calibration Data ◽

Ground Truth Data ◽

Depth Images ◽

Difficult Time ◽

Rgb Images ◽

Bounding Boxes ◽

Realistic Scenes ◽

Intrinsic And Extrinsic Parameters ◽

Realistic Data

With the progress of machine learning, the demand for realistic data with high-quality annotations has been thriving. In order to generalize well, considerable amounts of data are required, especially realistic ground-truth data, for tasks such as object detection and scene segmentation. Such data can be difficult, time-consuming, and expensive to collect. This article presents a dataset of household objects and box scenes commonly found in warehouse environments. The dataset was obtained using a robotic setup with four different cameras. It contains reconstructed objects and scenes, as well as raw RGB and depth images, camera poses, pixel-wise labels of objects directly in the RGB images, and 3D bounding boxes with poses in the world frame. Furthermore, raw calibration data are provided, together with the intrinsic and extrinsic parameters for all the sensors. By providing object labels as pixel-wise masks, 3D, and 2D object bounding boxes, this dataset is useful for both object recognition and instance segmentation. The realistic scenes provided will serve for learning-based algorithms applied to scenarios where boxes of objects are often found, such as in the logistics sector. Both the dataset and the tools for data processing are published and available online.

Download Full-text

YOLO MDE: Object Detection with Monocular Depth Estimation

Electronics ◽

10.3390/electronics11010076 ◽

2021 ◽

Vol 11 (1) ◽

pp. 76

Author(s):

Jongsub Yu ◽

Hyukdoo Choi

Keyword(s):

Risk Assessment ◽

Object Detection ◽

Network Architecture ◽

Ground Truth ◽

Depth Estimation ◽

Autonomous Driving ◽

Depth Prediction ◽

Bounding Box ◽

Monocular Depth ◽

Bounding Boxes

This paper presents an object detector with depth estimation using monocular camera images. Previous detection studies have typically focused on detecting objects with 2D or 3D bounding boxes. A 3D bounding box consists of the center point, its size parameters, and heading information. However, predicting complex output compositions leads a model to have generally low performances, and it is not necessary for risk assessment for autonomous driving. We focused on predicting a single depth per object, which is essential for risk assessment for autonomous driving. Our network architecture is based on YOLO v4, which is a fast and accurate one-stage object detector. We added an additional channel to the output layer for depth estimation. To train depth prediction, we extract the closest depth from the 3D bounding box coordinates of ground truth labels in the dataset. Our model is compared with the latest studies on 3D object detection using the KITTI object detection benchmark. As a result, we show that our model achieves higher detection performance and detection speed than existing models with comparable depth accuracy.

Download Full-text

1 year, 1000 km: The Oxford RobotCar dataset

The International Journal of Robotics Research ◽

10.1177/0278364916679498 ◽

2016 ◽

Vol 36 (1) ◽

pp. 3-15 ◽

Cited By ~ 264

Author(s):

Will Maddern ◽

Geoffrey Pascoe ◽

Chris Linegar ◽

Paul Newman

Keyword(s):

Autonomous Vehicles ◽

Ground Truth ◽

Heavy Rain ◽

Weather Conditions ◽

Autonomous Driving ◽

Urban Environments ◽

Ground Truth Data ◽

Full Dataset ◽

Localization And Mapping

We present a challenging new dataset for autonomous driving: the Oxford RobotCar Dataset. Over the period of May 2014 to December 2015 we traversed a route through central Oxford twice a week on average using the Oxford RobotCar platform, an autonomous Nissan LEAF. This resulted in over 1000 km of recorded driving with almost 20 million images collected from 6 cameras mounted to the vehicle, along with LIDAR, GPS and INS ground truth. Data was collected in all weather conditions, including heavy rain, night, direct sunlight and snow. Road and building works over the period of a year significantly changed sections of the route from the beginning to the end of data collection. By frequently traversing the same route over the period of a year we enable research investigating long-term localization and mapping for autonomous vehicles in real-world, dynamic urban environments. The full dataset is available for download at: http://robotcar-dataset.robots.ox.ac.uk

Download Full-text