scholarly journals What do deep saliency models learn about where we look in scenes?

2021 ◽  
Author(s):  
Taylor R. Hayes ◽  
John M. Henderson
Keyword(s):  
State Of The Art ◽  
Black Box ◽  
Image Features ◽  
Know How ◽  
Mixed Effects Modeling ◽  
Current State ◽  
Saliency Model ◽  
Image Saliency ◽  
High Level ◽  
Scene Features

Abstract Deep saliency models represent the current state-of-the-art for predicting where humans look in real-world scenes. However, for deep saliency models to inform cognitive theories of attention, we need to know how deep saliency models predict where people look. Here we open the black box of deep saliency models using an approach that models the association between the output of 3 prominent deep saliency models (MSI-Net, DeepGaze II, and SAM-ResNet) and low-, mid-, and high-level scene features. Specifically, we measured the association between each deep saliency model and low-level image saliency, mid-level contour symmetry and junctions, and high-level meaning by applying a mixed effects modeling approach to a large eye movement dataset. We found that despite different architectures, training regimens, and loss functions, all three deep saliency models were most strongly associated with high-level meaning. These findings suggest that deep saliency models are primarily learning image features associated with scene meaning.

scholarly journals Deep saliency models learn low-, mid-, and high-level features to predict scene attention

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Taylor R. Hayes ◽  
John M. Henderson
Keyword(s):  
Image Features ◽  
Interaction Patterns ◽  
Low Level ◽  
Mixed Effects Modeling ◽  
Current State ◽  
Saliency Model ◽  
Image Saliency ◽  
High Level ◽  
Scene Features ◽  
Benchmarking Performance

AbstractDeep saliency models represent the current state-of-the-art for predicting where humans look in real-world scenes. However, for deep saliency models to inform cognitive theories of attention, we need to know how deep saliency models prioritize different scene features to predict where people look. Here we open the black box of three prominent deep saliency models (MSI-Net, DeepGaze II, and SAM-ResNet) using an approach that models the association between attention, deep saliency model output, and low-, mid-, and high-level scene features. Specifically, we measured the association between each deep saliency model and low-level image saliency, mid-level contour symmetry and junctions, and high-level meaning by applying a mixed effects modeling approach to a large eye movement dataset. We found that all three deep saliency models were most strongly associated with high-level and low-level features, but exhibited qualitatively different feature weightings and interaction patterns. These findings suggest that prominent deep saliency models are primarily learning image features associated with high-level scene meaning and low-level image saliency and highlight the importance of moving beyond simply benchmarking performance.

scholarly journals A Survey of Graphical Page Object Detection with Deep Neural Networks

2021 ◽  
Author(s):  
Jwalin Bhatt ◽  
Khurram Azeem Hashmi ◽  
Muhammad Zeshan Afzal ◽  
Didier Stricker
Keyword(s):  
Deep Learning ◽  
Object Detection ◽  
Conceptual Understanding ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Learning Approaches ◽  
Document Images ◽  
Essential Information ◽  
Current State ◽  
High Level

In any document, graphical elements like tables, figures, and formulas contain essential information. The processing and interpretation of such information require specialized algorithms. Off-the-shelf OCR components cannot process this information reliably. Therefore, an essential step in document analysis pipelines is to detect these graphical components. It leads to a high-level conceptual understanding of the documents that makes digitization of documents viable. Since the advent of deep learning, the performance of deep learning-based object detection has improved many folds. In this work, we outline and summarize the deep learning approaches for detecting graphical page objects in the document images. Therefore, we discuss the most relevant deep learning-based approaches and state-of-the-art graphical page object detection in document images. This work provides a comprehensive understanding of the current state-of-the-art and related challenges. Furthermore, we discuss leading datasets along with the quantitative evaluation. Moreover, it discusses briefly the promising directions that can be utilized for further improvements.

scholarly journals Making rainfall features fun: scientific activities for teaching children aged 5–12 years

2016 ◽  
Vol 20 (5) ◽  
pp. 1751-1763 ◽  
Author(s):  
Auguste Gires ◽  
Catherine L. Muller ◽  
Marie-Agathe le Gueut ◽  
Daniel Schertzer
Keyword(s):  
Environmental Science ◽  
State Of The Art ◽  
Scientific Output ◽  
Pedagogical Practices ◽  
Research Projects ◽  
Scientific Culture ◽  
Children's Book ◽  
Current State ◽  
High Level ◽  
Rainy Days

Abstract. Research projects now rely on an array of different channels to increase impact, including high-level scientific output, tools, and equipment, but also communication, outreach, and educational activities. This paper focuses on education for children aged 5–12 years and presents activities that aim to help them (and their teachers) grasp some of the complex underlying issues in environmental science. More generally, it helps children to become familiarized with science and scientists, with the aim to enhance scientific culture and promote careers in this field. The activities developed are focused on rainfall: (a) designing and using a disdrometer to observe the variety of drop sizes; (b) careful recording of successive dry and rainy days and reproducing patterns using a simple model based on fractal random multiplicative cascades; and (c) collaboratively writing a children's book about rainfall. These activities are discussed in the context of current state-of-the-art pedagogical practices and goals set by project funders, especially in a European Union framework.

scholarly journals Deep Convolutional Neural Network for Pedestrian Detection with Multi-Levels Features Fusion

2018 ◽  
Vol 232 ◽  
pp. 01061
Author(s):  
Danhua Li ◽  
Xiaofeng Di ◽  
Xuan Qu ◽  
Yunfei Zhao ◽  
Honggang Kong
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
State Of The Art ◽  
Pedestrian Detection ◽  
Deep Convolutional Neural Network ◽  
High Quality ◽  
Low Level ◽  
Features Fusion ◽  
Current State ◽  
High Level

Pedestrian detection aims to localize and recognize every pedestrian instance in an image with a bounding box. The current state-of-the-art method is Faster RCNN, which is such a network that uses a region proposal network (RPN) to generate high quality region proposals, while Fast RCNN is used to classifiers extract features into corresponding categories. The contribution of this paper is integrated low-level features and high-level features into a Faster RCNN-based pedestrian detection framework, which efficiently increase the capacity of the feature. Through our experiments, we comprehensively evaluate our framework, on the Caltech pedestrian detection benchmark and our methods achieve state-of-the-art accuracy and present a competitive result on Caltech dataset.

scholarly journals A Review of Stochastic Programming Methods for Optimization of Process Systems Under Uncertainty

2021 ◽  
Vol 2 ◽  
Author(s):  
Can Li ◽  
Ignacio E. Grossmann
Keyword(s):  
Stochastic Programming ◽  
Systems Engineering ◽  
State Of The Art ◽  
Mathematical Framework ◽  
Process Systems Engineering ◽  
Process Industries ◽  
Process Systems ◽  
Current State ◽  
Engineering Problems ◽  
High Level

Uncertainties are widespread in the optimization of process systems, such as uncertainties in process technologies, prices, and customer demands. In this paper, we review the basic concepts and recent advances of a risk-neutral mathematical framework called “stochastic programming” and its applications in solving process systems engineering problems under uncertainty. This review intends to provide both a tutorial for beginners without prior experience and a high-level overview of the current state-of-the-art developments for experts in process systems engineering and stochastic programming. The mathematical formulations and algorithms for two-stage and multistage stochastic programming are reviewed with illustrative examples from process industries. The differences between stochastic programming under exogenous uncertainty and endogenous uncertainties are discussed. The concepts and several data-driven methods for generating scenario trees are also reviewed.

scholarly journals Gamificação e realidade aumentada em processos de ensino-aprendizagem: uma revisão sistemática da literatura

2021 ◽  
Vol 14 (2) ◽  
pp. 262-274
Author(s):  
Orlando Rosa Junior ◽  
Tiago De Oliveira ◽  
Ezequiel Zorzal
Keyword(s):  
Augmented Reality ◽  
State Of The Art ◽  
Learning Assessment ◽  
Assessment Methodology ◽  
Know How ◽  
Current State ◽  
Research Questions ◽  
Programming Knowledge ◽  
Specific Learning ◽  
Do So

This paper presents a Systematic Literature Review (RSL) on the use of Gamification and Augmented Reality applied in Education. RSL enabled mapping and knowledge of the current state of related studies. Thirty articles related to the state of the art were analyzed. From the analysis, it was found that there is no specific learning assessment methodology when applying the Augmented Reality and Gamification tools. An analysis of the results was made to answer the research questions. The study showed that lack of programming knowledge is also a factor that hinders the advancement of research since the researcher must know how to develop the application or have in his team someone who has the competence to do so. Finally, we present the comparatives and analyzes of the studies to obtain answers based on the research questions developed.

Modular Reconfigurable Robotics

2019 ◽  
Vol 2 (1) ◽  
pp. 63-88 ◽  
Author(s):  
Jungwon Seo ◽  
Jamie Paik ◽  
Mark Yim
Keyword(s):  
State Of The Art ◽  
Low Cost ◽  
Future Research ◽  
Research Directions ◽  
Current State ◽  
Robot Systems ◽  
Technical Issues ◽  
Future Research Directions ◽  
Reconfigurable Robot ◽  
High Level

This article reviews the current state of the art in the development of modular reconfigurable robot (MRR) systems and suggests promising future research directions. A wide variety of MRR systems have been presented to date, and these robots promise to be versatile, robust, and low cost compared with other conventional robot systems. MRR systems thus have the potential to outperform traditional systems with a fixed morphology when carrying out tasks that require a high level of flexibility. We begin by introducing the taxonomy of MRRs based on their hardware architecture. We then examine recent progress in the hardware and the software technologies for MRRs, along with remaining technical issues. We conclude with a discussion of open challenges and future research directions.

scholarly journals A Survey of Graphical Page Object Detection with Deep Neural Networks

2021 ◽  
Vol 11 (12) ◽  
pp. 5344
Author(s):  
Jwalin Bhatt ◽  
Khurram Azeem Hashmi ◽  
Muhammad Zeshan Afzal ◽  
Didier Stricker
Keyword(s):  
Deep Learning ◽  
Object Detection ◽  
Conceptual Understanding ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Learning Approaches ◽  
Document Images ◽  
Essential Information ◽  
Current State ◽  
High Level

In any document, graphical elements like tables, figures, and formulas contain essential information. The processing and interpretation of such information require specialized algorithms. Off-the-shelf OCR components cannot process this information reliably. Therefore, an essential step in document analysis pipelines is to detect these graphical components. It leads to a high-level conceptual understanding of the documents that make the digitization of documents viable. Since the advent of deep learning, deep learning-based object detection performance has improved many folds. This work outlines and summarizes the deep learning approaches for detecting graphical page objects in document images. Therefore, we discuss the most relevant deep learning-based approaches and state-of-the-art graphical page object detection in document images. This work provides a comprehensive understanding of the current state-of-the-art and related challenges. Furthermore, we discuss leading datasets along with the quantitative evaluation. Moreover, it discusses briefly the promising directions that can be utilized for further improvements.

scholarly journals Modelling Processes and Products in the Cereal Chain

Foods ◽  
2021 ◽  
Vol 10 (1) ◽  
pp. 82
Author(s):  
Otilia Carvalho ◽  
Maria N. Charalambides ◽  
Ilija Djekić ◽  
Christos Athanassiou ◽  
Serafim Bakalis ◽  
...  
Keyword(s):  
Supply Chain ◽  
Predictive Models ◽  
Food Processing ◽  
State Of The Art ◽  
Current State ◽  
The World ◽  
Modelling Techniques ◽  
One Step ◽  
High Level ◽  
Different Parts

In recent years, modelling techniques have become more frequently adopted in the field of food processing, especially for cereal-based products, which are among the most consumed foods in the world. Predictive models and simulations make it possible to explore new approaches and optimize proceedings, potentially helping companies reduce costs and limit carbon emissions. Nevertheless, as the different phases of the food processing chain are highly specialized, advances in modelling are often unknown outside of a single domain, and models rarely take into account more than one step. This paper introduces the first high-level overview of modelling techniques employed in different parts of the cereal supply chain, from farming to storage, from drying to milling, from processing to consumption. This review, issued from a networking project including researchers from over 30 different countries, aims at presenting the current state of the art in each domain, showing common trends and synergies, to finally suggest promising future venues for research.

scholarly journals Using Social Signals to Predict Shoplifting: A Transparent Approach to a Sensitive Activity Analysis Problem

Sensors ◽  
2021 ◽  
Vol 21 (20) ◽  
pp. 6812
Author(s):  
Shane Reid ◽  
Sonya Coleman ◽  
Philip Vance ◽  
Dermot Kerr ◽  
Siobhan O’Neill
Keyword(s):  
Signal Processing ◽  
State Of The Art ◽  
High Accuracy ◽  
Black Box ◽  
Social Signals ◽  
Social Signal Processing ◽  
Legal Cases ◽  
Current State ◽  
Social Signal ◽  
High Degree

Retail shoplifting is one of the most prevalent forms of theft and has accounted for over one billion GBP in losses for UK retailers in 2018. An automated approach to detecting behaviours associated with shoplifting using surveillance footage could help reduce these losses. Until recently, most state-of-the-art vision-based approaches to this problem have relied heavily on the use of black box deep learning models. While these models have been shown to achieve very high accuracy, this lack of understanding on how decisions are made raises concerns about potential bias in the models. This limits the ability of retailers to implement these solutions, as several high-profile legal cases have recently ruled that evidence taken from these black box methods is inadmissible in court. There is an urgent need to develop models which can achieve high accuracy while providing the necessary transparency. One way to alleviate this problem is through the use of social signal processing to add a layer of understanding in the development of transparent models for this task. To this end, we present a social signal processing model for the problem of shoplifting prediction which has been trained and validated using a novel dataset of manually annotated shoplifting videos. The resulting model provides a high degree of understanding and achieves accuracy comparable with current state of the art black box methods.

Sign in / Sign up

Export Citation Format

Share Document