Eye Gaze Correction Using Generative Adversarial Networks

2018 ◽  
Author(s):  
Takahiko Yamamoto ◽  
Masataka Seo ◽  
Toshihiko Kitajima ◽  
Yen-Wei Chen
Keyword(s):  
Eye Gaze ◽  
Generative Adversarial Networks ◽  
Adversarial Networks ◽  
Gaze Correction

scholarly journals Detecting performance difficulty of learners in colonoscopy: Evidence from eye-tracking

2021 ◽  
Vol 14 (2) ◽  
Author(s):  
Xin Liu ◽  
Bin Zheng ◽  
Xiaoqin Duan ◽  
Wenjing He ◽  
Yuandong Li ◽  
...  
Keyword(s):  
Deep Learning ◽  
Eye Tracking ◽  
Human Performance ◽  
Short Term Memory ◽  
Eye Gaze ◽  
Generative Adversarial Networks ◽  
Feeding Strategies ◽  
Human Eye ◽  
Adversarial Networks ◽  
Personal Feedback

Eye-tracking can help decode the intricate control mechanism in human performance. In healthcare, physicians-in-training requires extensive practice to improve their healthcare skills. When a trainee encounters any difficulty in the practice, they will need feedback from experts to improve their performance. The personal feedback is time-consuming and subjected to bias. In this study, we tracked the eye movements of trainees during their colonoscopic performance in simulation. We applied deep learning algorithms to detect the eye-tracking metrics on the moments of navigation lost (MNL), a signature sign for performance difficulty during colonoscopy. Basic human eye gaze and pupil characteristics were learned and verified by the deep convolutional generative adversarial networks (DCGANs); the generated data were fed to the Long Short-Term Memory (LSTM) networks with three different data feeding strategies to classify MNLs from the entire colonoscopic procedure. Outputs from deep learning were compared to the expert’s judgment on the MNLs based on colonoscopic videos. The best classification outcome was achieved when we fed human eye data with 1000 synthesized eye data, where accuracy (90%), sensitivity (90%), and specificity (88%) were optimized. This study built an important foundation for our work of developing a self-adaptive education system for training healthcare skills using simulation.

scholarly journals POV-Matter, cinematic POV and algorithmic POV between affects and umwelten

2019 ◽  
Vol 8 (1) ◽  
pp. 156-168 ◽  
Author(s):  
Mitra Azar
Keyword(s):  
Ecological Niche ◽  
Eye Gaze ◽  
Affect Theory ◽  
Point Of View ◽  
Generative Adversarial Networks ◽  
Adversarial Networks ◽  
Von Uexküll ◽  
Fundamental Categories ◽  
The Relationship ◽  
Do So

The essay refers to affect theory as a conceptual toolbox to draw a genealogy of POV (Point of View) that goes from the formation of the first organic POV to the reinvention of POV by the cinematic apparatus up to the latest development of algorithmic POV in machine vision and AI. The essay engages with Bergson’s conviction that there’s no perception without affection, and tests it against a phenomenological, cinematic and machinic notion of POV. To do so, the essay introduces what the German biologist Jacob von Uexküll has called Umwelt — the ecological niche emerging from the affordances between organisms, space, and (when applicable) technology. Furthermore, fundamental categories of both phenomenology and psycho-analysis are put at work in relation to cinematic POV and to the algorithmic POV produced by Generative Adversarial Networks (GANs), which seems to re-invent the relationship between seeing/seen (Merleau-Ponty) and eye/gaze (Lacan). This re-invention confirms the category of Umwelt and affect as markers for understanding the transformation between a phenomenological, cinematic and algorithmic notion of POV.

ORGANIC (1).pdf

2017 ◽  
Author(s):  
Benjamin Sanchez-Lengeling ◽  
Carlos Outeiral ◽  
Gabriel L. Guimaraes ◽  
Alan Aspuru-Guzik
Keyword(s):  
Machine Learning ◽  
Learning Community ◽  
Chemical Species ◽  
Material Design ◽  
Organic Photovoltaic ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Photovoltaic Material

Molecular discovery seeks to generate chemical species tailored to very specific needs. In this paper, we present ORGANIC, a framework based on Objective-Reinforced Generative Adversarial Networks (ORGAN), capable of producing a distribution over molecular space that matches with a certain set of desirable metrics. This methodology combines two successful techniques from the machine learning community: a Generative Adversarial Network (GAN), to create non-repetitive sensible molecular species, and Reinforcement Learning (RL), to bias this generative distribution towards certain attributes. We explore several applications, from optimization of random physicochemical properties to candidates for drug discovery and organic photovoltaic material design.

Sign in / Sign up

Export Citation Format

Share Document