Real-Time Obstacle Detection Based on Image Semantic Segmentation and Fusion Network

2021 ◽  
Vol 38 (2) ◽  
pp. 443-449
Author(s):  
Wei Liu
Keyword(s):  
Real Time ◽  
Semantic Segmentation ◽  
Fruit Production ◽  
Image Features ◽  
Obstacle Detection ◽  
Depth Image ◽  
High Definition ◽  
Real Time Detection ◽  
Under Performing ◽  
Set Up

During fruit production, the robots must walk stably across the orchard, and detect the obstacles in real time on its path. With the rapid process of deep convolutional neural network (CNN), it is now a hot topic to enable orchard robots to detect obstacles through image semantic segmentation. However, most such obstacle detection schemes are under performing in the complex environment of orchards. To solve the problem, this paper proposes an image semantic fusion network for real-time detection of small obstacles. Two branches were set up to extract features from red-green-blue (RGB) image and depth image, respectively. The information extracted by different modules were merged to complement the image features. The proposed network can operate rapidly, and support the real-time detection of obstacles by orchard robots. Experiments on orchard scenarios show that our network is superior to the latest image semantic segmentation methods, highly accurate in the recognition of high-definition images, and extremely fast in reasoning.

RSNet: Rail semantic segmentation network for extracting aerial railroad images

2021 ◽  
pp. 1-18
Author(s):  
R.S. Rampriya ◽  
Sabarinathan ◽  
R. Suganya
Keyword(s):  
Real Time ◽  
Visual Processing ◽  
Feature Fusion ◽  
Semantic Segmentation ◽  
Vital Role ◽  
Obstacle Detection ◽  
Aerial Images ◽  
Computationally Efficient ◽  
Fusion Algorithm ◽  
Uav Images

In the near future, combo of UAV (Unmanned Aerial Vehicle) and computer vision will play a vital role in monitoring the condition of the railroad periodically to ensure passenger safety. The most significant module involved in railroad visual processing is obstacle detection, in which caution is obstacle fallen near track gage inside or outside. This leads to the importance of detecting and segment the railroad as three key regions, such as gage inside, rails, and background. Traditional railroad segmentation methods depend on either manual feature selection or expensive dedicated devices such as Lidar, which is typically less reliable in railroad semantic segmentation. Also, cameras mounted on moving vehicles like a drone can produce high-resolution images, so segmenting precise pixel information from those aerial images has been challenging due to the railroad surroundings chaos. RSNet is a multi-level feature fusion algorithm for segmenting railroad aerial images captured by UAV and proposes an attention-based efficient convolutional encoder for feature extraction, which is robust and computationally efficient and modified residual decoder for segmentation which considers only essential features and produces less overhead with higher performance even in real-time railroad drone imagery. The network is trained and tested on a railroad scenic view segmentation dataset (RSSD), which we have built from real-time UAV images and achieves 0.973 dice coefficient and 0.94 jaccard on test data that exhibits better results compared to the existing approaches like a residual unit and residual squeeze net.

MobileNet-Yolo Based wildlife detection model: A case study in yunnan tongbiguan nature reserve, China

2021 ◽  
pp. 1-11
Author(s):  
Tingting Zhao ◽  
Xiaoli Yi ◽  
Zhiyong Zeng ◽  
Tao Feng
Keyword(s):  
Real Time ◽  
Nature Reserve ◽  
Complex Model ◽  
Training Data ◽  
Mean Average Precision ◽  
Detection Accuracy ◽  
High Definition ◽  
Average Precision ◽  
Data Set ◽  
Real Time Detection

YTNR (Yunnan Tongbiguan Nature Reserve) is located in the westernmost part of China’s tropical regions and is the only area in China with the tropical biota of the Irrawaddy River system. The reserve has abundant tropical flora and fauna resources. In order to realize the real-time detection of wild animals in this area, this paper proposes an improved YOLO (You only look once) network. The original YOLO model can achieve higher detection accuracy, but due to the complex model structure, it cannot achieve a faster detection speed on the CPU detection platform. Therefore, the lightweight network MobileNet is introduced to replace the backbone feature extraction network in YOLO, which realizes real-time detection on the CPU platform. In response to the difficulty in collecting wild animal image data, the research team deployed 50 high-definition cameras in the study area and conducted continuous observations for more than 1,000 hours. In the end, this research uses 1410 images of wildlife collected in the field and 1577 wildlife images from the internet to construct a research data set combined with the manual annotation of domain experts. At the same time, transfer learning is introduced to solve the problem of insufficient training data and the network is difficult to fit. The experimental results show that our model trained on a training set containing 2419 animal images has a mean average precision of 93.6% and an FPS (Frame Per Second) of 3.8 under the CPU. Compared with YOLO, the mean average precision is increased by 7.7%, and the FPS value is increased by 3.

scholarly journals Real-Time Fusion Network for RGB-D Semantic Segmentation Incorporating Unexpected Obstacle Detection for Road-Driving Images

2020 ◽  
Vol 5 (4) ◽  
pp. 5558-5565 ◽  
Author(s):  
Lei Sun ◽  
Kailun Yang ◽  
Xinxin Hu ◽  
Weijian Hu ◽  
Kaiwei Wang
Keyword(s):  
Real Time ◽  
Semantic Segmentation ◽  
Obstacle Detection

scholarly journals Implementation of a Lightweight Semantic Segmentation Algorithm in Road Obstacle Detection

Sensors ◽  
2020 ◽  
Vol 20 (24) ◽  
pp. 7089
Author(s):  
Bushi Liu ◽  
Yongbo Lv ◽  
Yang Gu ◽  
Wanjun Lv
Keyword(s):  
Real Time ◽  
Spatial Information ◽  
Feature Fusion ◽  
Semantic Segmentation ◽  
Spatial Location ◽  
Autonomous Driving ◽  
Obstacle Detection ◽  
Depth Information ◽  
Long Time ◽  
Deep Learning Network

Due to deep learning’s accurate cognition of the street environment, the convolutional neural network has achieved dramatic development in the application of street scenes. Considering the needs of autonomous driving and assisted driving, in a general way, computer vision technology is used to find obstacles to avoid collisions, which has made semantic segmentation a research priority in recent years. However, semantic segmentation has been constantly facing new challenges for quite a long time. Complex network depth information, large datasets, real-time requirements, etc., are typical problems that need to be solved urgently in the realization of autonomous driving technology. In order to address these problems, we propose an improved lightweight real-time semantic segmentation network, which is based on an efficient image cascading network (ICNet) architecture, using multi-scale branches and a cascaded feature fusion unit to extract rich multi-level features. In this paper, a spatial information network is designed to transmit more prior knowledge of spatial location and edge information. During the course of the training phase, we append an external loss function to enhance the learning process of the deep learning network system as well. This lightweight network can quickly perceive obstacles and detect roads in the drivable area from images to satisfy autonomous driving characteristics. The proposed model shows substantial performance on the Cityscapes dataset. With the premise of ensuring real-time performance, several sets of experimental comparisons illustrate that SP-ICNet enhances the accuracy of road obstacle detection and provides nearly ideal prediction outputs. Compared to the current popular semantic segmentation network, this study also demonstrates the effectiveness of our lightweight network for road obstacle detection in autonomous driving.

A REAL-TIME DETECTION ALGORITHM FOR VISION-BASED PEDESTRIAN PROTECTION

2008 ◽  
Vol 05 (01) ◽  
pp. 11-30 ◽  
Author(s):  
GUANGLIN MA ◽  
SU-BIRM PARK ◽  
ALEXANDER IOFFE ◽  
STEFAN MÜLLER-SCHNEIDERS ◽  
ANTON KUMMERT
Keyword(s):  
Real Time ◽  
Pedestrian Detection ◽  
Ground Plane ◽  
Detection Algorithm ◽  
Obstacle Detection ◽  
Low Contrast ◽  
The Road ◽  
Novel Approach ◽  
Real Time Detection ◽  
Flat Ground

This paper discusses the robust, real-time detection of stationary and moving pedestrians utilizing a single car-mounted monochrome camera. First, the system detects potential pedestrians above the ground plane by combining conventional Inverse Perspective Mapping (IPM)-based obstacle detection with the vertical 1D profile evaluation of the IPM detection result. Usage of the vertical profile increases the robustness of detection in low-contrast images as well as the detection of distant pedestrians significantly. A fast digital image stabilization algorithm is used to compensate for erroneous detections whenever the flat ground plane assumption is an inaccurate model of the road surface. Finally, a low-level pedestrian-oriented segmentation and fast symmetry search on the leg region of pedestrians is also presented. A novel approach termed Pedestrian Detection Strip (PDS) is used to improve the calculation time by a factor of six compared to conventional approaches.

ASSESSMENT OF NEW HIGH-DEFINITION BOREHOLE IMAGING-WHILE-DRILLING TECHNOLOGY: LEARNINGS FROM PRE-SALT CARBONATES OF BRAZIL

2021 ◽  
Author(s):  
Nicolas Orban ◽  
◽  
Shashank Garg ◽  
Mikhail Shaldaev ◽  
Chandramani Shrivastava ◽  
...  
Keyword(s):  
Real Time ◽  
Image Features ◽  
Added Value ◽  
Quality Data ◽  
Operational Parameters ◽  
Time Data ◽  
High Definition ◽  
Amplitude Data ◽  
Image Quality Control ◽  
Borehole Imaging

The pre-salt carbonates of Brazil pose drilling and characterization challenges associated with inherent reservoir heterogeneity; and borehole imaging while drilling often provides insights helpful for both, operational and subsequent decisions. The findings and learnings from a 3-well campaign, offshore Brazil are presented to assess and validate a recently deployed high-definition borehole imaging technology that provides industry’s first real-time ultrasonic amplitude images and time-to-depth corrections for best possible images maintaining the geological features integrity. High-definition ultrasonic measurements were acquired at two central frequencies with 0.2-in resolution and provided amplitude and transit time images for geological characterization and petrophysical evaluation in addition to azimuthal ultrasonic calipers. The lossy nature of amplitude data makes it difficult to transmit in real-time; therefore, a unique data compression technology was used to achieve industry’s first high quality amplitude images streaming while drilling. In deepwater operations acquisition of high-definition logging while drilling (LWD) images can be severely degraded if time-to-depth offset due to heave is not compensated. Recently developed heave-filtering workflows ensured the integrity of subsurface features. The time-indexed data was processed with this application in real-time, providing good results and confidence in the capability of this technology. Image-logs of the first well were helpful in interpretation and added value to the reservoir understanding; however, many intervals suffered from lack of confidence in image features. Simulations were performed to improve the images acquisition parameters based on learnings from this experience. New optimized operational parameters were applied in next two wells, resulting in image logs of excellent quality. Data from second well suffered from high heave while drilling, which required implementation of the heave-filtering memory data workflow. For the third well, an additional requirement for real-time image quality-control was defined, requiring data to be processed after every drill-stand. Real-time data quality provided confidence in optimal quality of memory data, thereby eliminating the need of post-drilling wireline operations in open-hole. The images acquired in memory helped characterize intervals of stromatolites with various morphology, and zones of vugs distribution, providing excellent alternative for wireline logging, de-risking the operations in pre-salt carbonate logging in Brazil offshore operations.

scholarly journals 64 Seizing an Opportunity: Setting Up A Virtual Wetlab During the COVID19 Pandemic

2021 ◽  
Vol 108 (Supplement_6) ◽  
Author(s):  
L Holland ◽  
I Ahmed
Keyword(s):  
Cardiac Surgery ◽  
Real Time ◽  
Real Life ◽  
Skills Training ◽  
High Definition ◽  
Online Platform ◽  
Training Tool ◽  
Set Up ◽  
The Uk ◽  
Coronary Anastomosis

Abstract Aim The COVID19 pandemic has negatively impacted surgical training across the globe. In the UK, educational events and training days were cancelled for trainees at all levels. However, these unusual times provide opportunity to develop novel teaching methods. Our aim was to design a “virtual cardiac surgery wetlab” that allows real-time skills training. Method With support from industry partners Edwards Lifesciences, Wetlab and Connexon365 (Webinar Hosting Platform) all the necessary materials were distributed. A package was sent to trainer and trainees that included microinstruments, sutures, synthetic vessels and valve models, high-definition camera, along with printed instructions and video to outline set-up of the rig. Trainees only need to provide is a computer. A consultant surgeon demonstrated how to perform a coronary anastomosis and aortic valve replacement and was then able to supervise trainees and provide real-time feedback using the online platform. Results Feedback from trainer and trainees has shown that the system is an easy-to-set-up, effective training tool for teaching cardiac surgery skills including coronary anastomosis and valve surgery. The online platform enabled good vision and engagement between trainer and trainee and also enabled recording of the session. Distributing all of the materials nationwide means the virtual wetlab can take place anywhere, including in the comfort of home, for both trainee and trainer. Conclusions Surgical skills training can still take place effectively in the absence of “real life” wetlabs. Given the current COVID19 pandemic continues to interrupt traditional surgical teaching, this method has potential to be adopted nationwide to minimise the disruption.

Sign in / Sign up

Export Citation Format

Share Document