scholarly journals LiCaNet: Further Enhancement of Joint Perception and Motion Prediction based on Multi-Modal Fusion

2021 ◽  
Author(s):  
Yasser Khalil ◽  
Hussein T. Mouftah
Keyword(s):  
Prediction Models ◽  
Comprehensive Evaluation ◽  
State Of The Art ◽  
Autonomous Driving ◽  
Motion Prediction ◽  
Camera Network ◽  
Considerable Effort ◽  
Safety And Reliability ◽  
Accurate Perception ◽  
Camera Sensor

<p>The safety and reliability of autonomous driving pivots on the accuracy of perception and motion prediction pipelines, which in turn reckons primarily on the sensors deployed onboard. Slight confusion in perception and motion prediction can result in catastrophic consequences due to misinterpretation in later pipelines. Therefore, researchers have recently devoted considerable effort towards developing accurate perception and motion prediction models. To that end, we propose LIDAR Camera network (LiCaNet) that leverages multi-modal fusion to further enhance the joint perception and motion prediction performance accomplished in our earlier work. LiCaNet expands on our previous fusion network by adding a camera image to the fusion of RV image with historical BEV data sourced from a LIDAR sensor. We present a comprehensive evaluation to validate the outstanding performance of LiCaNet compared to the state-of-the-art. Experiments reveal that utilizing a camera sensor results in a substantial perception gain over our previous fusion network and a steep reduction in displacement errors. Moreover, the majority of the achieved improvement falls within camera range, with the highest registered for small and distant objects, confirming the significance of incorporating a camera sensor into a fusion network.</p>

scholarly journals LiCaNet: Further Enhancement of Joint Perception and Motion Prediction based on Multi-Modal Fusion

2021 ◽  
Author(s):  
Yasser Khalil ◽  
Hussein T. Mouftah
Keyword(s):  
Prediction Models ◽  
Comprehensive Evaluation ◽  
State Of The Art ◽  
Autonomous Driving ◽  
Motion Prediction ◽  
Camera Network ◽  
Considerable Effort ◽  
Safety And Reliability ◽  
Accurate Perception ◽  
Camera Sensor

<p>The safety and reliability of autonomous driving pivots on the accuracy of perception and motion prediction pipelines, which in turn reckons primarily on the sensors deployed onboard. Slight confusion in perception and motion prediction can result in catastrophic consequences due to misinterpretation in later pipelines. Therefore, researchers have recently devoted considerable effort towards developing accurate perception and motion prediction models. To that end, we propose LIDAR Camera network (LiCaNet) that leverages multi-modal fusion to further enhance the joint perception and motion prediction performance accomplished in our earlier work. LiCaNet expands on our previous fusion network by adding a camera image to the fusion of RV image with historical BEV data sourced from a LIDAR sensor. We present a comprehensive evaluation to validate the outstanding performance of LiCaNet compared to the state-of-the-art. Experiments reveal that utilizing a camera sensor results in a substantial perception gain over our previous fusion network and a steep reduction in displacement errors. Moreover, the majority of the achieved improvement falls within camera range, with the highest registered for small and distant objects, confirming the significance of incorporating a camera sensor into a fusion network.</p>

Persistent memory hash indexes

2021 ◽  
Vol 14 (5) ◽  
pp. 785-798
Author(s):  
Daokun Hu ◽  
Zhiwen Chen ◽  
Jianbing Wu ◽  
Jianhua Sun ◽  
Hao Chen
Keyword(s):  
Future Development ◽  
High Performance ◽  
Performance Metrics ◽  
Comprehensive Evaluation ◽  
State Of The Art ◽  
Hash Tables ◽  
Trade Offs ◽  
Depth Analysis ◽  
Persistent Memory ◽  
Memory Modules

Persistent memory (PM) is increasingly being leveraged to build hash-based indexing structures featuring cheap persistence, high performance, and instant recovery, especially with the recent release of Intel Optane DC Persistent Memory Modules. However, most of them are evaluated on DRAM-based emulators with unreal assumptions, or focus on the evaluation of specific metrics with important properties sidestepped. Thus, it is essential to understand how well the proposed hash indexes perform on real PM and how they differentiate from each other if a wider range of performance metrics are considered. To this end, this paper provides a comprehensive evaluation of persistent hash tables. In particular, we focus on the evaluation of six state-of-the-art hash tables including Level hashing, CCEH, Dash, PCLHT, Clevel, and SOFT, with real PM hardware. Our evaluation was conducted using a unified benchmarking framework and representative workloads. Besides characterizing common performance properties, we also explore how hardware configurations (such as PM bandwidth, CPU instructions, and NUMA) affect the performance of PM-based hash tables. With our in-depth analysis, we identify design trade-offs and good paradigms in prior arts, and suggest desirable optimizations and directions for the future development of PM-based hash tables.

scholarly journals One Proxy Device Is Enough for Hardware-Aware Neural Architecture Search

2021 ◽  
Vol 5 (3) ◽  
pp. 1-34
Author(s):  
Bingqian Lu ◽  
Jianyi Yang ◽  
Weiwen Jiang ◽  
Yiyu Shi ◽  
Shaolei Ren
Keyword(s):  
State Of The Art ◽  
Autonomous Driving ◽  
Pareto Optimal ◽  
Video Content ◽  
Fast Evaluation ◽  
Video Content Analysis ◽  
Search Spaces ◽  
Neural Architecture ◽  
Real World Applications ◽  
Prohibitive Cost

Convolutional neural networks (CNNs) are used in numerous real-world applications such as vision-based autonomous driving and video content analysis. To run CNN inference on various target devices, hardware-aware neural architecture search (NAS) is crucial. A key requirement of efficient hardware-aware NAS is the fast evaluation of inference latencies in order to rank different architectures. While building a latency predictor for each target device has been commonly used in state of the art, this is a very time-consuming process, lacking scalability in the presence of extremely diverse devices. In this work, we address the scalability challenge by exploiting latency monotonicity --- the architecture latency rankings on different devices are often correlated. When strong latency monotonicity exists, we can re-use architectures searched for one proxy device on new target devices, without losing optimality. In the absence of strong latency monotonicity, we propose an efficient proxy adaptation technique to significantly boost the latency monotonicity. Finally, we validate our approach and conduct experiments with devices of different platforms on multiple mainstream search spaces, including MobileNet-V2, MobileNet-V3, NAS-Bench-201, ProxylessNAS and FBNet. Our results highlight that, by using just one proxy device, we can find almost the same Pareto-optimal architectures as the existing per-device NAS, while avoiding the prohibitive cost of building a latency predictor for each device.

scholarly journals ConnectIt

2020 ◽  
Vol 14 (4) ◽  
pp. 653-667
Author(s):  
Laxman Dhulipala ◽  
Changwan Hong ◽  
Julian Shun
Keyword(s):  
Experimental Evaluation ◽  
Comprehensive Evaluation ◽  
State Of The Art ◽  
Graph Connectivity ◽  
Connected Components ◽  
Sampling Strategies ◽  
Spanning Forest ◽  
Speed Up ◽  
Minimum Spanning Forest ◽  
Edge Sampling

Connected components is a fundamental kernel in graph applications. The fastest existing multicore algorithms for solving graph connectivity are based on some form of edge sampling and/or linking and compressing trees. However, many combinations of these design choices have been left unexplored. In this paper, we design the ConnectIt framework, which provides different sampling strategies as well as various tree linking and compression schemes. ConnectIt enables us to obtain several hundred new variants of connectivity algorithms, most of which extend to computing spanning forest. In addition to static graphs, we also extend ConnectIt to support mixes of insertions and connectivity queries in the concurrent setting. We present an experimental evaluation of ConnectIt on a 72-core machine, which we believe is the most comprehensive evaluation of parallel connectivity algorithms to date. Compared to a collection of state-of-the-art static multicore algorithms, we obtain an average speedup of 12.4x (2.36x average speedup over the fastest existing implementation for each graph). Using ConnectIt, we are able to compute connectivity on the largest publicly-available graph (with over 3.5 billion vertices and 128 billion edges) in under 10 seconds using a 72-core machine, providing a 3.1x speedup over the fastest existing connectivity result for this graph, in any computational setting. For our incremental algorithms, we show that our algorithms can ingest graph updates at up to several billion edges per second. To guide the user in selecting the best variants in ConnectIt for different situations, we provide a detailed analysis of the different strategies. Finally, we show how the techniques in ConnectIt can be used to speed up two important graph applications: approximate minimum spanning forest and SCAN clustering.

scholarly journals Mosaic Super-resolution via Sequential Feature Pyramid Networks

2019 ◽  
Author(s):  
Mehrdad Shoeiby ◽  
Mohammad Ali Armin ◽  
Sadegh Aliakbarian ◽  
Saeed Anwar ◽  
Lars petersson
Keyword(s):  
State Of The Art ◽  
Super Resolution ◽  
Autonomous Driving ◽  
Single Shot ◽  
Current State ◽  
Wide Range ◽  
Feature Pyramid ◽  
Novel Method ◽  
Convolutional Lstm ◽  
Mosaic Images

<div>Advances in the design of multi-spectral cameras have</div><div>led to great interests in a wide range of applications, from</div><div>astronomy to autonomous driving. However, such cameras</div><div>inherently suffer from a trade-off between the spatial and</div><div>spectral resolution. In this paper, we propose to address</div><div>this limitation by introducing a novel method to carry out</div><div>super-resolution on raw mosaic images, multi-spectral or</div><div>RGB Bayer, captured by modern real-time single-shot mo-</div><div>saic sensors. To this end, we design a deep super-resolution</div><div>architecture that benefits from a sequential feature pyramid</div><div>along the depth of the network. This, in fact, is achieved</div><div>by utilizing a convolutional LSTM (ConvLSTM) to learn the</div><div>inter-dependencies between features at different receptive</div><div>fields. Additionally, by investigating the effect of different</div><div>attention mechanisms in our framework, we show that a</div><div>ConvLSTM inspired module is able to provide superior at-</div><div>tention in our context. Our extensive experiments and anal-</div><div>yses evidence that our approach yields significant super-</div><div>resolution quality, outperforming current state-of-the-art</div><div>mosaic super-resolution methods on both Bayer and multi-</div><div>spectral images. Additionally, to the best of our knowledge,</div><div>our method is the first specialized method to super-resolve</div><div>mosaic images, whether it be multi-spectral or Bayer.</div><div><br></div>

scholarly journals On the Development of Autonomous Vehicle Safety Distance by an RSS Model Based on a Variable Focus Function Camera

Sensors ◽  
2021 ◽  
Vol 21 (20) ◽  
pp. 6733
Author(s):  
Min-Joong Kim ◽  
Sung-Hun Yu ◽  
Tong-Hyun Kim ◽  
Joo-Uk Kim ◽  
Young-Min Kim
Keyword(s):  
Autonomous Vehicles ◽  
Focal Length ◽  
Autonomous Vehicle ◽  
Autonomous Driving ◽  
Cruise Control ◽  
Radar Sensors ◽  
Safety And Reliability ◽  
Time Required ◽  
Variable Focus ◽  
Camera Sensors

Today, a lot of research on autonomous driving technology is being conducted, and various vehicles with autonomous driving functions, such as ACC (adaptive cruise control) are being released. The autonomous vehicle recognizes obstacles ahead by the fusion of data from various sensors, such as lidar and radar sensors, including camera sensors. As the number of vehicles equipped with such autonomous driving functions increases, securing safety and reliability is a big issue. Recently, Mobileye proposed the RSS (responsibility-sensitive safety) model, which is a white box mathematical model, to secure the safety of autonomous vehicles and clarify responsibility in the case of an accident. In this paper, a method of applying the RSS model to a variable focus function camera that can cover the recognition range of a lidar sensor and a radar sensor with a single camera sensor is considered. The variables of the RSS model suitable for the variable focus function camera were defined, the variable values were determined, and the safe distances for each velocity were derived by applying the determined variable values. In addition, as a result of considering the time required to obtain the data, and the time required to change the focal length of the camera, it was confirmed that the response time obtained using the derived safe distance was a valid result.

scholarly journals Demand Prediction with Machine Learning Models; State of the Art and a Systematic Review of Advances

2019 ◽  
Author(s):  
Amir Mosavi ◽  
Sina Faizollahzadeh Ardabili ◽  
Shahabodin Shamshirband
Keyword(s):  
Machine Learning ◽  
Energy Production ◽  
Production Management ◽  
Prediction Models ◽  
State Of The Art ◽  
Electricity Demand ◽  
Original Research ◽  
Review Of The Literature ◽  
Future Prospects ◽  
Demand Prediction

Electricity demand prediction is vital for energy production management and proper exploitation of the present resources. Recently, several novel machine learning (ML) models have been employed for electricity demand prediction to estimate the future prospects of the energy requirements. The main objective of this study is to review the various ML models applied for electricity demand prediction. Through a novel search and taxonomy, the most relevant original research articles in the field are identified and further classified according to the ML modeling technique, perdition type, and the application area. A comprehensive review of the literature identifies the major ML models, their applications and a discussion on the evaluation of their performance. This paper further makes a discussion on the trend and the performance of the ML models. As the result, this research reports an outstanding rise in the accuracy, robustness, precision and the generalization ability of the prediction models using the hybrid and ensemble ML algorithms.

scholarly journals Flood Prediction Using Machine Learning Models: Literature Review

Water ◽  
2018 ◽  
Vol 10 (11) ◽  
pp. 1536 ◽  
Author(s):  
Amir Mosavi ◽  
Pinar Ozturk ◽  
Kwok-wing Chau
Keyword(s):  
Machine Learning ◽  
Prediction Models ◽  
Comprehensive Evaluation ◽  
Human Life ◽  
Cost Effective ◽  
Performance Comparison ◽  
Flood Prediction ◽  
Effective Strategies ◽  
Hybridization Data ◽  
Prediction Systems

Floods are among the most destructive natural disasters, which are highly complex to model. The research on the advancement of flood prediction models contributed to risk reduction, policy suggestion, minimization of the loss of human life, and reduction of the property damage associated with floods. To mimic the complex mathematical expressions of physical processes of floods, during the past two decades, machine learning (ML) methods contributed highly in the advancement of prediction systems providing better performance and cost-effective solutions. Due to the vast benefits and potential of ML, its popularity dramatically increased among hydrologists. Researchers through introducing novel ML methods and hybridizing of the existing ones aim at discovering more accurate and efficient prediction models. The main contribution of this paper is to demonstrate the state of the art of ML models in flood prediction and to give insight into the most suitable models. In this paper, the literature where ML models were benchmarked through a qualitative analysis of robustness, accuracy, effectiveness, and speed are particularly investigated to provide an extensive overview on the various ML algorithms used in the field. The performance comparison of ML models presents an in-depth understanding of the different techniques within the framework of a comprehensive evaluation and discussion. As a result, this paper introduces the most promising prediction methods for both long-term and short-term floods. Furthermore, the major trends in improving the quality of the flood prediction models are investigated. Among them, hybridization, data decomposition, algorithm ensemble, and model optimization are reported as the most effective strategies for the improvement of ML methods. This survey can be used as a guideline for hydrologists as well as climate scientists in choosing the proper ML method according to the prediction task.

scholarly journals DeepPurpose: a deep learning library for drug–target interaction prediction

2020 ◽  
Author(s):  
Kexin Huang ◽  
Tianfan Fu ◽  
Lucas M Glass ◽  
Marinka Zitnik ◽  
Cao Xiao ◽  
...  
Keyword(s):  
Deep Learning ◽  
Drug Target ◽  
Prediction Models ◽  
State Of The Art ◽  
Supplementary Information ◽  
Target Interaction ◽  
Interaction Prediction ◽  
Computer Scientists ◽  
Benchmark Datasets ◽  
Biomedical Field

Abstract Summary Accurate prediction of drug–target interactions (DTI) is crucial for drug discovery. Recently, deep learning (DL) models for show promising performance for DTI prediction. However, these models can be difficult to use for both computer scientists entering the biomedical field and bioinformaticians with limited DL experience. We present DeepPurpose, a comprehensive and easy-to-use DL library for DTI prediction. DeepPurpose supports training of customized DTI prediction models by implementing 15 compound and protein encoders and over 50 neural architectures, along with providing many other useful features. We demonstrate state-of-the-art performance of DeepPurpose on several benchmark datasets. Availability and implementation https://github.com/kexinhuang12345/DeepPurpose. Contact [email protected] Supplementary information Supplementary data are available at Bioinformatics online.

Sign in / Sign up

Export Citation Format

Share Document