scholarly journals Lane and Road Marker Semantic Video Segmentation Using Mask Cropping and Optical Flow Estimation

Sensors ◽  
2021 ◽  
Vol 21 (21) ◽  
pp. 7156
Author(s):  
Guansheng Xing ◽  
Ziming Zhu
Keyword(s):  
Optical Flow ◽  
Video Segmentation ◽  
Learning Algorithm ◽  
Time Consistency ◽  
Autonomous Driving ◽  
Target Area ◽  
Temporal Consistency ◽  
Current Frame ◽  
The Past ◽  
Single Output

Lane and road marker segmentation is crucial in autonomous driving, and many related methods have been proposed in this field. However, most of them are based on single-frame prediction, which causes unstable results between frames. Some semantic multi-frame segmentation methods produce error accumulation and are not fast enough. Therefore, we propose a deep learning algorithm that takes into account the continuity information of adjacent image frames, including image sequence processing and an end-to-end trainable multi-input single-output network to jointly process the segmentation of lanes and road markers. In order to emphasize the location of the target with high probability in the adjacent frames and to refine the segmentation result of the current frame, we explicitly consider the time consistency between frames, expand the segmentation region of the previous frame, and use the optical flow of the adjacent frames to reverse the past prediction, then use it as an additional input of the network in training and reasoning, thereby improving the network’s attention to the target area of the past frame. We segmented lanes and road markers on the Baidu Apolloscape lanemark segmentation dataset and CULane dataset, and present benchmarks for different networks. The experimental results show that this method accelerates the segmentation speed of video lanes and road markers by 2.5 times, increases accuracy by 1.4%, and reduces temporal consistency by only 2.2% at most.

scholarly journals Every Frame Counts: Joint Learning of Video Segmentation and Optical Flow

2020 ◽  
Vol 34 (07) ◽  
pp. 10713-10720
Author(s):  
Mingyu Ding ◽  
Zhe Wang ◽  
Bolei Zhou ◽  
Jianping Shi ◽  
Zhiwu Lu ◽  
...  
Keyword(s):  
Optical Flow ◽  
Video Segmentation ◽  
Video Clip ◽  
Semantic Segmentation ◽  
Temporal Consistency ◽  
Flow Estimation ◽  
Optical Flow Estimation ◽  
Optical Flows ◽  
Benchmark Datasets ◽  
Spatio Temporal

A major challenge for video semantic segmentation is the lack of labeled data. In most benchmark datasets, only one frame of a video clip is annotated, which makes most supervised methods fail to utilize information from the rest of the frames. To exploit the spatio-temporal information in videos, many previous works use pre-computed optical flows, which encode the temporal consistency to improve the video segmentation. However, the video segmentation and optical flow estimation are still considered as two separate tasks. In this paper, we propose a novel framework for joint video semantic segmentation and optical flow estimation. Semantic segmentation brings semantic information to handle occlusion for more robust optical flow estimation, while the non-occluded optical flow provides accurate pixel-level temporal correspondences to guarantee the temporal consistency of the segmentation. Moreover, our framework is able to utilize both labeled and unlabeled frames in the video through joint training, while no additional calculation is required in inference. Extensive experiments show that the proposed model makes the video semantic segmentation and optical flow estimation benefit from each other and outperforms existing methods under the same settings in both tasks.

scholarly journals HYPERPROPERTIES-BASED OPTICAL FLOW-BASED AUTONOMOUS DRIVING SYSTEM

2017 ◽  
Vol 10 (13) ◽  
pp. 254
Author(s):  
Ankush Rai ◽  
Jagadeesh Kanna R
Keyword(s):  
Optical Flow ◽  
Autonomous Driving ◽  
Feature Points ◽  
Motion Vectors ◽  
Driving System ◽  
The Past ◽  
Non Linear ◽  
Autonomous Driving System ◽  
Constraint Sets

This study presents an autonomous driving system based on the principles of trace vectors derived from hyperproperty of a modified optical flowalgorithm. This technique allows keeping track of the past motion vectors by tracking the constraint sets to overcome the non-linear attributes ofthe deformable feature points and motion vectors. The results presented in this work exhibits stable tracking and multi-step prediction in a limitednumber of steps with less training vectors.

scholarly journals Dynamic Warping Network for Semantic Video Segmentation

Complexity ◽  
2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Jiangyun Li ◽  
Yikai Zhao ◽  
Xingjian He ◽  
Xinxin Zhu ◽  
Jing Liu
Keyword(s):  
Optical Flow ◽  
Video Sequence ◽  
Video Segmentation ◽  
State Of The Art ◽  
Temporal Consistency ◽  
Feature Maps ◽  
Spatiotemporal Information ◽  
Benchmark Datasets ◽  
Additional Calculation ◽  
Dynamic Warping

A major challenge for semantic video segmentation is how to exploit the spatiotemporal information and produce consistent results for a video sequence. Many previous works utilize the precomputed optical flow to warp the feature maps across adjacent frames. However, the imprecise optical flow and the warping operation without any learnable parameters may not achieve accurate feature warping and only bring a slight improvement. In this paper, we propose a novel framework named Dynamic Warping Network (DWNet) to adaptively warp the interframe features for improving the accuracy of warping-based models. Firstly, we design a flow refinement module (FRM) to optimize the precomputed optical flow. Then, we propose a flow-guided convolution (FG-Conv) to achieve the adaptive feature warping based on the refined optical flow. Furthermore, we introduce the temporal consistency loss including the feature consistency loss and prediction consistency loss to explicitly supervise the warped features instead of simple feature propagation and fusion, which guarantees the temporal consistency of video segmentation. Note that our DWNet adopts extra constraints to improve the temporal consistency in the training phase, while no additional calculation and postprocessing are required during inference. Extensive experiments show that our DWNet can achieve consistent improvement over various strong baselines and achieves state-of-the-art accuracy on the Cityscapes and CamVid benchmark datasets.

scholarly journals Predicting Fraud Victimization Using Classical Machine Learning

Entropy ◽  
2021 ◽  
Vol 23 (3) ◽  
pp. 300
Author(s):  
Mark Lokanan ◽  
Susan Liu
Keyword(s):  
Machine Learning ◽  
Financial Literacy ◽  
Learning Algorithm ◽  
Demographic Characteristics ◽  
Financial Knowledge ◽  
The Past ◽  
Machine Learning Model ◽  
Long Time ◽  
Regulatory Organization ◽  
Investment Fraud

Protecting financial consumers from investment fraud has been a recurring problem in Canada. The purpose of this paper is to predict the demographic characteristics of investors who are likely to be victims of investment fraud. Data for this paper came from the Investment Industry Regulatory Organization of Canada’s (IIROC) database between January of 2009 and December of 2019. In total, 4575 investors were coded as victims of investment fraud. The study employed a machine-learning algorithm to predict the probability of fraud victimization. The machine learning model deployed in this paper predicted the typical demographic profile of fraud victims as investors who classify as female, have poor financial knowledge, know the advisor from the past, and are retired. Investors who are characterized as having limited financial literacy but a long-time relationship with their advisor have reduced probabilities of being victimized. However, male investors with low or moderate-level investment knowledge were more likely to be preyed upon by their investment advisors. While not statistically significant, older adults, in general, are at greater risk of being victimized. The findings from this paper can be used by Canadian self-regulatory organizations and securities commissions to inform their investors’ protection mandates.

scholarly journals End-to-End Autonomous Driving Through Dueling Double Deep Q-Network

2021 ◽  
Author(s):  
Baiyu Peng ◽  
Qi Sun ◽  
Shengbo Eben Li ◽  
Dongsuk Kum ◽  
Yuming Yin ◽  
...  
Keyword(s):  
Neural Network ◽  
State Space ◽  
Learning Algorithm ◽  
Rapid Development ◽  
Autonomous Driving ◽  
Saliency Map ◽  
Hierarchical Architecture ◽  
Link Type ◽  
The Neural Network ◽  
End To End

AbstractRecent years have seen the rapid development of autonomous driving systems, which are typically designed in a hierarchical architecture or an end-to-end architecture. The hierarchical architecture is always complicated and hard to design, while the end-to-end architecture is more promising due to its simple structure. This paper puts forward an end-to-end autonomous driving method through a deep reinforcement learning algorithm Dueling Double Deep Q-Network, making it possible for the vehicle to learn end-to-end driving by itself. This paper firstly proposes an architecture for the end-to-end lane-keeping task. Unlike the traditional image-only state space, the presented state space is composed of both camera images and vehicle motion information. Then corresponding dueling neural network structure is introduced, which reduces the variance and improves sampling efficiency. Thirdly, the proposed method is applied to The Open Racing Car Simulator (TORCS) to demonstrate its great performance, where it surpasses human drivers. Finally, the saliency map of the neural network is visualized, which indicates the trained network drives by observing the lane lines. A video for the presented work is available online, https://youtu.be/76ciJmIHMD8 or https://v.youku.com/v_show/id_XNDM4ODc0MTM4NA==.html.

scholarly journals Novel AI driven approach to classify infant motor functions

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Simon Reich ◽  
Dajie Zhang ◽  
Tomas Kulvicius ◽  
Sven Bölte ◽  
Karin Nielsen-Saines ◽  
...  
Keyword(s):  
Learning Algorithm ◽  
Movement Pattern ◽  
Video Stream ◽  
Typically Developing ◽  
The Past ◽  
Movement Assessment ◽  
Computer Based ◽  
Ablation Study ◽  
First Time ◽  
Efficient Screening

AbstractThe past decade has evinced a boom of computer-based approaches to aid movement assessment in early infancy. Increasing interests have been dedicated to develop AI driven approaches to complement the classic Prechtl general movements assessment (GMA). This study proposes a novel machine learning algorithm to detect an age-specific movement pattern, the fidgety movements (FMs), in a prospectively collected sample of typically developing infants. Participants were recorded using a passive, single camera RGB video stream. The dataset of 2800 five-second snippets was annotated by two well-trained and experienced GMA assessors, with excellent inter- and intra-rater reliabilities. Using OpenPose, the infant full pose was recovered from the video stream in the form of a 25-points skeleton. This skeleton was used as input vector for a shallow multilayer neural network (SMNN). An ablation study was performed to justify the network’s architecture and hyperparameters. We show for the first time that the SMNN is sufficient to discriminate fidgety from non-fidgety movements in a sample of age-specific typical movements with a classification accuracy of 88%. The computer-based solutions will complement original GMA to consistently perform accurate and efficient screening and diagnosis that may become universally accessible in daily clinical practice in the future.

scholarly journals IDENTIFYING OVERLAPPED OBJECTS FOR VIDEO INDEXING AND MODELING IN MULTIMEDIA DATABASE SYSTEMS

2001 ◽  
Vol 10 (04) ◽  
pp. 715-734 ◽  
Author(s):  
SHU-CHING CHEN ◽  
MEI-LING SHYU ◽  
CHENGCUI ZHANG ◽  
R. L. KASHYAP
Keyword(s):  
Spatial Information ◽  
Video Segmentation ◽  
Database Systems ◽  
Video Indexing ◽  
Video Data ◽  
Multimedia Database ◽  
Current Frame ◽  
Data Indexing ◽  
Spatio Temporal ◽  
Input Strings

The identification of the overlapped objects is a great challenge in object tracking and video data indexing. For this purpose, a backtrack-chain-updation split algorithm is proposed to assist an unsupervised video segmentation method called the "simultaneous partition and class parameter estimation" (SPCPE) algorithm to identify the overlapped objects in the video sequence. The backtrack-chain-updation split algorithm can identify the split segment (object) and use the information in the current frame to update the previous frames in a backtrack-chain manner. The split algorithm provides more accurate temporal and spatial information of the semantic objects so that the semantic objects can be indexed and modeled by multimedia input strings and the multimedia augmented transition network (MATN) model. The MATN model is based on the ATN model that has been used in artificial intelligence (AI) areas for natural language understanding systems, and its inputs are modeled by the multimedia input strings. In this paper, we will show that the SPCPE algorithm together with the backtrack-chain-updation split algorithm can significantly enhance the efficiency of spatio-temporal video indexing by improving the accuracy of multimedia database queries related to semantic objects.

scholarly journals Comparison of Machine Learning algorithm for COVID-19 Death Risk Prediction

2021 ◽  
Author(s):  
Praveeen Anandhanathan ◽  
Priyanka Gopalan
Keyword(s):  
Machine Learning ◽  
Learning Algorithm ◽  
Machine Learning Techniques ◽  
Support Vector ◽  
Nearest Neighbour ◽  
Decision Tree Algorithm ◽  
The Past ◽  
Random Forest Method ◽  
Learning Techniques ◽  
The World

Abstract Coronavirus disease (COVID-19) is spreading across the world. Since at first it has appeared in Wuhan, China in December 2019, it has become a serious issue across the globe. There are no accurate resources to predict and find the disease. So, by knowing the past patients’ records, it could guide the clinicians to fight against the pandemic. Therefore, for the prediction of healthiness from symptoms Machine learning techniques can be implemented. From this we are going to analyse only the symptoms which occurs in every patient. These predictions can help clinicians in the easier manner to cure the patients. Already for prediction of many of the diseases, techniques like SVM (Support vector Machine), Fuzzy k-Means Clustering, Decision Tree algorithm, Random Forest Method, ANN (Artificial Neural Network), KNN (k-Nearest Neighbour), Naïve Bayes, Linear Regression model are used. As we haven’t faced this disease before, we can’t say which technique will give the maximum accuracy. So, we are going to provide an efficient result by comparing all the such algorithms in RStudio.

scholarly journals Segmentation of Affected Crops using Deep Learning

2019 ◽  
Vol 8 (5S3) ◽  
pp. 232-234
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Monitoring System ◽  
Learning Algorithm ◽  
Target Area ◽  
Learning Technology ◽  
Machine Learning Algorithm ◽  
Normal Human ◽  
Human Monitoring ◽  
Disease Attack

Deep Learning technology can accurately predict the presence of diseases and pests in the agricultural farms. Upon this Machine learning algorithm, we can even predict accurately the chance of any disease and pest attacks in future For spraying the correct amount of fertilizer/pesticide to elimate host, the normal human monitoring system unable to predict accurately the total amount and ardent of pest and disease attack in farm. At the specified target area the artificial percepton tells the value accurately and give corrective measure and amount of fertilizers/ pesticides to be sprayed.

Sign in / Sign up

Export Citation Format

Share Document