Neural scene representation and rendering

Science ◽  
2018 ◽  
Vol 360 (6394) ◽  
pp. 1204-1210 ◽  
Author(s):  
S. M. Ali Eslami ◽  
Danilo Jimenez Rezende ◽  
Frederic Besse ◽  
Fabio Viola ◽  
Ari S. Morcos ◽  
...  
Keyword(s):  
Neural Networks ◽  
Open Problem ◽  
Domain Knowledge ◽  
Internal Representation ◽  
Representation Learning ◽  
Sensory Data ◽  
The World ◽  
Scene Representation ◽  
Intelligent Behavior ◽  
Important Open Problem

Scene representation—the process of converting visual sensory data into concise descriptions—is a requirement for intelligent behavior. Recent work has shown that neural networks excel at this task when provided with large, labeled datasets. However, removing the reliance on human labeling remains an important open problem. To this end, we introduce the Generative Query Network (GQN), a framework within which machines learn to represent scenes using only their own sensors. The GQN takes as input images of a scene taken from different viewpoints, constructs an internal representation, and uses this representation to predict the appearance of that scene from previously unobserved viewpoints. The GQN demonstrates representation learning without human labels or domain knowledge, paving the way toward machines that autonomously learn to understand the world around them.

scholarly journals Ontology, neural networks, and the social sciences

Synthese ◽  
2020 ◽  
Author(s):  
David Strohmaier
Keyword(s):  
Social Sciences ◽  
Neural Networks ◽  
Statistical Models ◽  
Internal Representation ◽  
Representation Learning ◽  
Machine Learning Techniques ◽  
Social Scientists ◽  
Social Phenomena ◽  
Ontological Commitments ◽  
The Social

AbstractThe ontology of social objects and facts remains a field of continued controversy. This situation complicates the life of social scientists who seek to make predictive models of social phenomena. For the purposes of modelling a social phenomenon, we would like to avoid having to make any controversial ontological commitments. The overwhelming majority of models in the social sciences, including statistical models, are built upon ontological assumptions that can be questioned. Recently, however, artificial neural networks (ANNs) have made their way into the social sciences, raising the question whether they can avoid controversial ontological assumptions. ANNs are largely distinguished from other statistical and machine learning techniques by being a representation-learning technique. That is, researchers can let the neural networks select which features of the data to use for internal representation instead of imposing their preconceptions. On this basis, I argue that neural networks can avoid ontological assumptions to a greater degree than common statistical models in the social sciences. I then go on, however, to establish that ANNs are not ontologically innocent either. The use of ANNs in the social sciences introduces ontological assumptions typically in at least two ways, via the input and via the architecture.

scholarly journals Encouraging an appropriate representation simplifies training of neural networks

2020 ◽  
Vol 12 (1) ◽  
pp. 102-111
Author(s):  
Krisztian Buza
Keyword(s):  
Neural Networks ◽  
Domain Knowledge ◽  
State Of The Art ◽  
Internal Representation ◽  
The State ◽  
Training Algorithm ◽  
Common Assumption ◽  
Generalization Ability ◽  
End To End

AbstractA common assumption about neural networks is that they can learn an appropriate internal representation on their own, see e.g. end-to-end learning. In this work we challenge this assumption. We consider two simple tasks and show that the state-of-the-art training algorithm fails, although the model itself is able to represent an appropriate solution. We will demonstrate that encouraging an appropriate internal representation allows the same model to solve these tasks. While we do not claim that it is impossible to solve these tasks by other means (such as neural networks with more layers), our results illustrate that integration of domain knowledge in form of a desired internal representation may improve the generalization ability of neural networks.

scholarly journals A Survey of Reinforcement Learning Informed by Natural Language

2019 ◽  
Author(s):  
Jelena Luketina ◽  
Nantas Nardelli ◽  
Gregory Farquhar ◽  
Jakob Foerster ◽  
Jacob Andreas ◽  
...  
Keyword(s):  
Decision Making ◽  
Reinforcement Learning ◽  
Natural Language ◽  
Language Processing ◽  
Domain Knowledge ◽  
Representation Learning ◽  
World Knowledge ◽  
Language Understanding ◽  
Text Corpora ◽  
The World

To be successful in real-world tasks, Reinforcement Learning (RL) needs to exploit the compositional, relational, and hierarchical structure of the world, and learn to transfer it to the task at hand. Recent advances in representation learning for language make it possible to build models that acquire world knowledge from text corpora and integrate this knowledge into downstream decision making problems. We thus argue that the time is right to investigate a tight integration of natural language understanding into RL in particular. We survey the state of the field, including work on instruction following, text games, and learning from textual domain knowledge. Finally, we call for the development of new environments as well as further investigation into the potential uses of recent Natural Language Processing (NLP) techniques for such tasks.

scholarly journals Deep Malaria Parasite Detection in Thin Blood Smear Microscopic Images

2021 ◽  
Vol 11 (5) ◽  
pp. 2284
Author(s):  
Asma Maqsood ◽  
Muhammad Shahid Farid ◽  
Muhammad Hassan Khan ◽  
Marcin Grzegorzek
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Red Blood Cells ◽  
Convolutional Neural Networks ◽  
Blood Cells ◽  
Superior Performance ◽  
Learning Models ◽  
Infected Female ◽  
The World ◽  
Augmentation Techniques

Malaria is a disease activated by a type of microscopic parasite transmitted from infected female mosquito bites to humans. Malaria is a fatal disease that is endemic in many regions of the world. Quick diagnosis of this disease will be very valuable for patients, as traditional methods require tedious work for its detection. Recently, some automated methods have been proposed that exploit hand-crafted feature extraction techniques however, their accuracies are not reliable. Deep learning approaches modernize the world with their superior performance. Convolutional Neural Networks (CNN) are vastly scalable for image classification tasks that extract features through hidden layers of the model without any handcrafting. The detection of malaria-infected red blood cells from segmented microscopic blood images using convolutional neural networks can assist in quick diagnosis, and this will be useful for regions with fewer healthcare experts. The contributions of this paper are two-fold. First, we evaluate the performance of different existing deep learning models for efficient malaria detection. Second, we propose a customized CNN model that outperforms all observed deep learning models. It exploits the bilateral filtering and image augmentation techniques for highlighting features of red blood cells before training the model. Due to image augmentation techniques, the customized CNN model is generalized and avoids over-fitting. All experimental evaluations are performed on the benchmark NIH Malaria Dataset, and the results reveal that the proposed algorithm is 96.82% accurate in detecting malaria from the microscopic blood smears.

Learning About the World by Learning About Images

2021 ◽  
pp. 096372142199033
Author(s):  
Katherine R. Storrs ◽  
Roland W. Fleming
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Visual Processing ◽  
Visual Experience ◽  
Natural Images ◽  
Sensory Coding ◽  
The World ◽  
Unsupervised Deep Learning ◽  
Statistical Regularities ◽  
Sensory Encoding

One of the deepest insights in neuroscience is that sensory encoding should take advantage of statistical regularities. Humans’ visual experience contains many redundancies: Scenes mostly stay the same from moment to moment, and nearby image locations usually have similar colors. A visual system that knows which regularities shape natural images can exploit them to encode scenes compactly or guess what will happen next. Although these principles have been appreciated for more than 60 years, until recently it has been possible to convert them into explicit models only for the earliest stages of visual processing. But recent advances in unsupervised deep learning have changed that. Neural networks can be taught to compress images or make predictions in space or time. In the process, they learn the statistical regularities that structure images, which in turn often reflect physical objects and processes in the outside world. The astonishing accomplishments of unsupervised deep learning reaffirm the importance of learning statistical regularities for sensory coding and provide a coherent framework for how knowledge of the outside world gets into visual cortex.

Model-Independent Control of a Flexible-Joint Robot Manipulator

2009 ◽  
Vol 131 (4) ◽  
Author(s):  
Withit Chatlatanagulchai ◽  
Peter H. Meckl
Keyword(s):  
Neural Networks ◽  
Open Problem ◽  
High Performance ◽  
Robot Manipulator ◽  
Intrinsic Property ◽  
Variable Structure ◽  
System Model ◽  
Control Input ◽  
Control Laws ◽  
Flexible Joint

Flexibility at the joint of a manipulator is an intrinsic property. Even “rigid-joint” robots, in fact, possess a certain amount of flexibility. Previous experiments confirmed that joint flexibility should be explicitly included in the model when designing a high-performance controller for a manipulator because the flexibility, if not dealt with, can excite system natural frequencies and cause severe damage. However, control design for a flexible-joint robot manipulator is still an open problem. Besides being described by a complicated system model for which the passivity property does not hold, the manipulator is also underactuated, that is, the control input does not drive the link directly, but through the flexible dynamics. Our work offers another possible solution to this open problem. We use three-layer neural networks to represent the system model. Their weights are adapted in real time and from scratch, which means we do not need the mathematical model of the robot in our control algorithm. All uncertainties are handled by variable-structure control. Backstepping structure allows input efforts to be applied to each subsystem where they are needed. Control laws to adjust all adjustable parameters are devised using Lyapunov’s second method to ensure that error trajectories are globally uniformly ultimately bounded. We present two state-feedback schemes: first, when neural networks are used to represent the unknown plant, and second, when neural networks are used to represent the unknown parts of the control laws. In the former case, we also design an observer to enable us to design a control law using only output signals—the link positions. We use simulations to compare our algorithms with some other well-known techniques. We use experiments to demonstrate the practicality of our algorithms.

Sign in / Sign up

Export Citation Format

Share Document