scholarly journals Smart Glass System Using Deep Learning for the Blind and Visually Impaired

Electronics ◽  
2021 ◽  
Vol 10 (22) ◽  
pp. 2756
Author(s):  
Mukhriddin Mukhiddinov ◽  
Jinsoo Cho
Keyword(s):  
Computer Vision ◽  
Deep Learning ◽  
Object Detection ◽  
Visually Impaired ◽  
Glass System ◽  
Salient Object ◽  
Audio Feedback ◽  
Low Light ◽  
Detection Model ◽  
Blind And Visually Impaired

Individuals suffering from visual impairments and blindness encounter difficulties in moving independently and overcoming various problems in their routine lives. As a solution, artificial intelligence and computer vision approaches facilitate blind and visually impaired (BVI) people in fulfilling their primary activities without much dependency on other people. Smart glasses are a potential assistive technology for BVI people to aid in individual travel and provide social comfort and safety. However, practically, the BVI are unable move alone, particularly in dark scenes and at night. In this study we propose a smart glass system for BVI people, employing computer vision techniques and deep learning models, audio feedback, and tactile graphics to facilitate independent movement in a night-time environment. The system is divided into four models: a low-light image enhancement model, an object recognition and audio feedback model, a salient object detection model, and a text-to-speech and tactile graphics generation model. Thus, this system was developed to assist in the following manner: (1) enhancing the contrast of images under low-light conditions employing a two-branch exposure-fusion network; (2) guiding users with audio feedback using a transformer encoder–decoder object detection model that can recognize 133 categories of sound, such as people, animals, cars, etc., and (3) accessing visual information using salient object extraction, text recognition, and refreshable tactile display. We evaluated the performance of the system and achieved competitive performance on the challenging Low-Light and ExDark datasets.

scholarly journals Salient Object Detection Techniques in Computer Vision—A Survey

Entropy ◽  
2020 ◽  
Vol 22 (10) ◽  
pp. 1174
Author(s):  
Ashish Kumar Gupta ◽  
Ayan Seal ◽  
Mukesh Prasad ◽  
Pritee Khanna
Keyword(s):  
Computer Vision ◽  
Deep Learning ◽  
Object Detection ◽  
Large Scale ◽  
Research Work ◽  
Salient Object Detection ◽  
Future Research ◽  
Automatic Identification ◽  
Salient Object ◽  
Detection Techniques

Detection and localization of regions of images that attract immediate human visual attention is currently an intensive area of research in computer vision. The capability of automatic identification and segmentation of such salient image regions has immediate consequences for applications in the field of computer vision, computer graphics, and multimedia. A large number of salient object detection (SOD) methods have been devised to effectively mimic the capability of the human visual system to detect the salient regions in images. These methods can be broadly categorized into two categories based on their feature engineering mechanism: conventional or deep learning-based. In this survey, most of the influential advances in image-based SOD from both conventional as well as deep learning-based categories have been reviewed in detail. Relevant saliency modeling trends with key issues, core techniques, and the scope for future research work have been discussed in the context of difficulties often faced in salient object detection. Results are presented for various challenging cases for some large-scale public datasets. Different metrics considered for assessment of the performance of state-of-the-art salient object detection models are also covered. Some future directions for SOD are presented towards end.

Saliency detection in deep learning era: trends of development

2019 ◽  
pp. 10-36 ◽  
Author(s):  
M. N. Favorskaya ◽  
L. C. Jain
Keyword(s):  
Deep Learning ◽  
Object Detection ◽  
Event Detection ◽  
Visual Analysis ◽  
Saliency Detection ◽  
Salient Object Detection ◽  
Public Image ◽  
Detection Methods ◽  
Salient Object ◽  
Salient Event

Introduction:Saliency detection is a fundamental task of computer vision. Its ultimate aim is to localize the objects of interest that grab human visual attention with respect to the rest of the image. A great variety of saliency models based on different approaches was developed since 1990s. In recent years, the saliency detection has become one of actively studied topic in the theory of Convolutional Neural Network (CNN). Many original decisions using CNNs were proposed for salient object detection and, even, event detection.Purpose:A detailed survey of saliency detection methods in deep learning era allows to understand the current possibilities of CNN approach for visual analysis conducted by the human eyes’ tracking and digital image processing.Results:A survey reflects the recent advances in saliency detection using CNNs. Different models available in literature, such as static and dynamic 2D CNNs for salient object detection and 3D CNNs for salient event detection are discussed in the chronological order. It is worth noting that automatic salient event detection in durable videos became possible using the recently appeared 3D CNN combining with 2D CNN for salient audio detection. Also in this article, we have presented a short description of public image and video datasets with annotated salient objects or events, as well as the often used metrics for the results’ evaluation.Practical relevance:This survey is considered as a contribution in the study of rapidly developed deep learning methods with respect to the saliency detection in the images and videos.

scholarly journals Overview of deep-learning based methods for salient object detection in videos

2020 ◽  
Vol 104 ◽  
pp. 107340
Author(s):  
Qiong Wang ◽  
Lu Zhang ◽  
Yan Li ◽  
Kidiyo Kpalma
Keyword(s):  
Deep Learning ◽  
Object Detection ◽  
Salient Object Detection ◽  
Salient Object

Prediction of Crime Scene Objects Using Deep Learning

2021 ◽  
Vol 9 (VIII) ◽  
pp. 857-864
Author(s):  
Vibhavari B Rao
Keyword(s):  
Deep Learning ◽  
Object Detection ◽  
Computer Architecture ◽  
Real Time ◽  
Surveillance System ◽  
Crime Scene ◽  
Detection Model ◽  
Detection Algorithms ◽  
Performance Metric ◽  
Smart Surveillance

The crime rates today can inevitably put a civilian's life in danger. While consistent efforts are being made to alleviate crime, there is also a dire need to create a smart and proactive surveillance system. Our project implements a smart surveillance system that would alert the authorities in real-time when a crime is being committed. During armed robberies and hostage situations, most often, the police cannot reach the place on time to prevent it from happening, owing to the lag in communication between the informants of the crime scene and the police. We propose an object detection model that implements deep learning algorithms to detect objects of violence such as pistols, knives, rifles from video surveillance footage, and in turn send real-time alerts to the authorities. There are a number of object detection algorithms being developed, each being evaluated under the performance metric mAP. On implementing Faster R-CNN with ResNet 101 architecture we found the mAP score to be about 91%. However, the downside to this is the excessive training and inferencing time it incurs. On the other hand, YOLOv5 architecture resulted in a model that performed very well in terms of speed. Its training speed was found to be 0.012 s / image during training but naturally, the accuracy was not as high as Faster R-CNN. With good computer architecture, it can run at about 40 fps. Thus, there is a tradeoff between speed and accuracy and it's important to strike a balance. We use transfer learning to improve accuracy by training the model on our custom dataset. This project can be deployed on any generic CCTV camera by setting up a live RTSP (real-time streaming protocol) and streaming the footage on a laptop or desktop where the deep learning model is being run.

Deep Learning Based Active Monitoring for Anti-collision between Vessels and Bridges

2019 ◽  
Author(s):  
Limu Chen ◽  
Ye Xia ◽  
Dexiong Pan ◽  
Chengbin Wang
Keyword(s):  
Decision Making ◽  
Deep Learning ◽  
Object Detection ◽  
Large Scale ◽  
Data Augmentation ◽  
Information Support ◽  
Single Shot ◽  
Active Monitoring ◽  
Detection Model ◽  
Comparison Results

<p>Deep-learning based navigational object detection is discussed with respect to active monitoring system for anti-collision between vessel and bridge. Motion based object detection method widely used in existing anti-collision monitoring systems is incompetent in dealing with complicated and changeable waterway for its limitations in accuracy, robustness and efficiency. The video surveillance system proposed contains six modules, including image acquisition, detection, tracking, prediction, risk evaluation and decision-making, and the detection module is discussed in detail. A vessel-exclusive dataset with tons of image samples is established for neural network training and a SSD (Single Shot MultiBox Detector) based object detection model with both universality and pertinence is generated attributing to tactics of sample filtering, data augmentation and large-scale optimization, which make it capable of stable and intelligent vessel detection. Comparison results with conventional methods indicate that the proposed deep-learning method shows remarkable advantages in robustness, accuracy, efficiency and intelligence. In-situ test is carried out at Songpu Bridge in Shanghai, and the results illustrate that the method is qualified for long-term monitoring and providing information support for further analysis and decision making.</p>

Indoor Navigation Aid Systems for the Blind and Visually Impaired Based on Depth Sensors

2021 ◽  
pp. 187-223
Author(s):  
Fernando Merchan ◽  
Martin Poveda ◽  
Danilo E. Cáceres-Hernández ◽  
Javier E. Sanchez-Galan
Keyword(s):  
Deep Learning ◽  
Visually Impaired ◽  
Indoor Navigation ◽  
Assistive Technologies ◽  
Microsoft Kinect ◽  
Depth Sensors ◽  
Blind And Visually Impaired ◽  
Recent Developments ◽  
Depth Vision

This chapter focuses on the contributions made in the development of assistive technologies for the navigation of blind and visually impaired (BVI) individuals. A special interest is placed on vision-based systems that make use of image (RGB) and depth (D) information to assist their indoor navigation. Many commercial RGB-D cameras exist on the market, but for many years the Microsoft Kinect has been used as a tool for research in this field. Therefore, first-hand experience and advances on the use of Kinect for the development of an indoor navigation aid system for BVI individuals is presented. Limitations that can be encountered in building such a system are addressed at length. Finally, an overview of novel avenues of research in indoor navigation for BVI individuals such as integration of computer vision algorithms, deep learning for the classification of objects, and recent developments with stereo depth vision are discussed.

scholarly journals SuperVAE: Superpixelwise Variational Autoencoder for Salient Object Detection

2019 ◽  
Vol 33 ◽  
pp. 8569-8576 ◽  
Author(s):  
Bo Li ◽  
Zhengxing Sun ◽  
Yuqi Guo
Keyword(s):  
Deep Learning ◽  
Object Detection ◽  
Saliency Detection ◽  
Salient Object Detection ◽  
Salient Object ◽  
Image Saliency ◽  
Spatial Consistency ◽  
Variational Autoencoder ◽  
Benchmark Datasets ◽  
Supervised Methods

Image saliency detection has recently witnessed rapid progress due to deep neural networks. However, there still exist many important problems in the existing deep learning based methods. Pixel-wise convolutional neural network (CNN) methods suffer from blurry boundaries due to the convolutional and pooling operations. While region-based deep learning methods lack spatial consistency since they deal with each region independently. In this paper, we propose a novel salient object detection framework using a superpixelwise variational autoencoder (SuperVAE) network. We first use VAE to model the image background and then separate salient objects from the background through the reconstruction residuals. To better capture semantic and spatial contexts information, we also propose a perceptual loss to take advantage from deep pre-trained CNNs to train our SuperVAE network. Without the supervision of mask-level annotated data, our method generates high quality saliency results which can better preserve object boundaries and maintain the spatial consistency. Extensive experiments on five wildly-used benchmark datasets show that the proposed method achieves superior or competitive performance compared to other algorithms including the very recent state-of-the-art supervised methods.

Sign in / Sign up

Export Citation Format

Share Document