Pictogram Generator from Korean Sentences using Emoticon and Saliency Map

For the fast detection and recognition of apple fruit targets, based on the real-time DeepSnake deep learning instance segmentation model, this paper provided an algorithm basis for the practical application and promotion of apple picking robots. Since the initial detection results have an important impact on the subsequent edge prediction, this paper proposed an automatic detection method for apple fruit targets in natural environments based on saliency detection and traditional color difference methods. Combined with the original image, the histogram backprojection algorithm was used to further optimize the salient image results. A dynamic adaptive overlapping target separation algorithm was proposed to locate the single target fruit and further to determine the initial contour for DeepSnake, in view of the possible overlapping fruit regions in the saliency map. Finally, the target fruit was labeled based on the segmentation results of the examples. In the experiment, 300 training datasets were used to train the DeepSnake model, and the self-built dataset containing 1036 pictures of apples in various situations under natural environment was tested. The detection accuracy of target fruits under non-overlapping shaded fruits, overlapping fruits, shaded branches and leaves, and poor illumination conditions were 99.12%, 94.78%, 90.71%, and 94.46% respectively. The comprehensive detection accuracy was 95.66%, and the average processing time was 0.42 s in 1036 test images, which showed that the proposed algorithm can effectively separate the overlapping fruits through a not-very-large training samples and realize the rapid and accurate detection of apple targets.

Download Full-text

Explaining Neural Networks Using Attentive Knowledge Distillation

Sensors ◽

10.3390/s21041280 ◽

2021 ◽

Vol 21 (4) ◽

pp. 1280

Author(s):

Hyeonseok Lee ◽

Sungchan Kim

Keyword(s):

Neural Networks ◽

Model Prediction ◽

Saliency Map ◽

Model Parameters ◽

Learning Capability ◽

Fine Grained ◽

Network Layers ◽

Saliency Maps ◽

Novel Approach ◽

Knowledge Distillation

Explaining the prediction of deep neural networks makes the networks more understandable and trusted, leading to their use in various mission critical tasks. Recent progress in the learning capability of networks has primarily been due to the enormous number of model parameters, so that it is usually hard to interpret their operations, as opposed to classical white-box models. For this purpose, generating saliency maps is a popular approach to identify the important input features used for the model prediction. Existing explanation methods typically only use the output of the last convolution layer of the model to generate a saliency map, lacking the information included in intermediate layers. Thus, the corresponding explanations are coarse and result in limited accuracy. Although the accuracy can be improved by iteratively developing a saliency map, this is too time-consuming and is thus impractical. To address these problems, we proposed a novel approach to explain the model prediction by developing an attentive surrogate network using the knowledge distillation. The surrogate network aims to generate a fine-grained saliency map corresponding to the model prediction using meaningful regional information presented over all network layers. Experiments demonstrated that the saliency maps are the result of spatially attentive features learned from the distillation. Thus, they are useful for fine-grained classification tasks. Moreover, the proposed method runs at the rate of 24.3 frames per second, which is much faster than the existing methods by orders of magnitude.

Download Full-text

Interpretable Saliency Map for Deep Reinforcement Learning

Journal of Physics Conference Series ◽

10.1088/1742-6596/1757/1/012075 ◽

2021 ◽

Vol 1757 (1) ◽

pp. 012075

Author(s):

Hong Zheng ◽

Yinglong Dai ◽

Fumin Yu ◽

Yuezhen Hu

Keyword(s):

Reinforcement Learning ◽

Saliency Map

Download Full-text

End-to-End Autonomous Driving Through Dueling Double Deep Q-Network

Automotive Innovation ◽

10.1007/s42154-021-00151-3 ◽

2021 ◽

Author(s):

Baiyu Peng ◽

Qi Sun ◽

Shengbo Eben Li ◽

Dongsuk Kum ◽

Yuming Yin ◽

...

Keyword(s):

Neural Network ◽

State Space ◽

Learning Algorithm ◽

Rapid Development ◽

Autonomous Driving ◽

Saliency Map ◽

Hierarchical Architecture ◽

Link Type ◽

The Neural Network ◽

End To End

AbstractRecent years have seen the rapid development of autonomous driving systems, which are typically designed in a hierarchical architecture or an end-to-end architecture. The hierarchical architecture is always complicated and hard to design, while the end-to-end architecture is more promising due to its simple structure. This paper puts forward an end-to-end autonomous driving method through a deep reinforcement learning algorithm Dueling Double Deep Q-Network, making it possible for the vehicle to learn end-to-end driving by itself. This paper firstly proposes an architecture for the end-to-end lane-keeping task. Unlike the traditional image-only state space, the presented state space is composed of both camera images and vehicle motion information. Then corresponding dueling neural network structure is introduced, which reduces the variance and improves sampling efficiency. Thirdly, the proposed method is applied to The Open Racing Car Simulator (TORCS) to demonstrate its great performance, where it surpasses human drivers. Finally, the saliency map of the neural network is visualized, which indicates the trained network drives by observing the lane lines. A video for the presented work is available online, https://youtu.be/76ciJmIHMD8 or https://v.youku.com/v_show/id_XNDM4ODc0MTM4NA==.html.

Download Full-text

Infrared Dim-small Object Detection Algorithm Based on Saliency Map Combined with Target Motion Feature

2020 IEEE International Conference on Progress in Informatics and Computing (PIC) ◽

10.1109/pic50277.2020.9350820 ◽

2020 ◽

Author(s):

WenWen Zhang ◽

ZhiChao Lian

Keyword(s):

Object Detection ◽

Detection Algorithm ◽

Saliency Map ◽

Target Motion ◽

Small Object ◽

Motion Feature ◽

Small Object Detection

Download Full-text

Saliency-Based Gaze Visualization for Eye Movement Analysis

Sensors ◽

10.3390/s21155178 ◽

2021 ◽

Vol 21 (15) ◽

pp. 5178

Author(s):

Sangbong Yoo ◽

Seongmin Jeong ◽

Seokyeon Kim ◽

Yun Jang

Keyword(s):

Eye Movements ◽

Visual Attention ◽

Eye Movement ◽

Visual Stimuli ◽

Saliency Map ◽

Gaze Behavior ◽

The Gaze ◽

Movement Data ◽

Visual Clues ◽

Human Visual Attention

Gaze movement and visual stimuli have been utilized to analyze human visual attention intuitively. Gaze behavior studies mainly show statistical analyses of eye movements and human visual attention. During these analyses, eye movement data and the saliency map are presented to the analysts as separate views or merged views. However, the analysts become frustrated when they need to memorize all of the separate views or when the eye movements obscure the saliency map in the merged views. Therefore, it is not easy to analyze how visual stimuli affect gaze movements since existing techniques focus excessively on the eye movement data. In this paper, we propose a novel visualization technique for analyzing gaze behavior using saliency features as visual clues to express the visual attention of an observer. The visual clues that represent visual attention are analyzed to reveal which saliency features are prominent for the visual stimulus analysis. We visualize the gaze data with the saliency features to interpret the visual attention. We analyze the gaze behavior with the proposed visualization to evaluate that our approach to embedding saliency features within the visualization supports us to understand the visual attention of an observer.

Download Full-text