Soft robot perception using embedded soft sensors and recurrent neural networks

2019 ◽  
Vol 4 (26) ◽  
pp. eaav1488 ◽  
Author(s):  
Thomas George Thuruthel ◽  
Benjamin Shih ◽  
Cecilia Laschi ◽  
Michael Thomas Tolley
Keyword(s):  
Ground Truth ◽  
Human Robot Interaction ◽  
Synthetic Analog ◽  
Soft Sensors ◽  
Biologically Inspired ◽  
Soft Systems ◽  
Robot Perception ◽  
Machine Learning Approach ◽  
Applied Forces ◽  
Deformation Models

Recent work has begun to explore the design of biologically inspired soft robots composed of soft, stretchable materials for applications including the handling of delicate materials and safe interaction with humans. However, the solid-state sensors traditionally used in robotics are unable to capture the high-dimensional deformations of soft systems. Embedded soft resistive sensors have the potential to address this challenge. However, both the soft sensors—and the encasing dynamical system—often exhibit nonlinear time-variant behavior, which makes them difficult to model. In addition, the problems of sensor design, placement, and fabrication require a great deal of human input and previous knowledge. Drawing inspiration from the human perceptive system, we created a synthetic analog. Our synthetic system builds models using a redundant and unstructured sensor topology embedded in a soft actuator, a vision-based motion capture system for ground truth, and a general machine learning approach. This allows us to model an unknown soft actuated system. We demonstrate that the proposed approach is able to model the kinematics of a soft continuum actuator in real time while being robust to sensor nonlinearities and drift. In addition, we show how the same system can estimate the applied forces while interacting with external objects. The role of action in perception is also presented. This approach enables the development of force and deformation models for soft robotic systems, which can be useful for a variety of applications, including human-robot interaction, soft orthotics, and wearable robotics.

Humans and humanoids

2018 ◽  
Author(s):  
Giorgio Metta
Keyword(s):  
Humanoid Robot ◽  
Robot Vision ◽  
Physical Space ◽  
Humanoid Robots ◽  
Human Robot Interaction ◽  
Biologically Inspired ◽  
Robot Interaction ◽  
Humanoid Robotics ◽  
Biomimetic Robot ◽  
Compliant Actuation

This chapter outlines a number of research lines that, starting from the observation of nature, attempt to mimic human behavior in humanoid robots. Humanoid robotics is one of the most exciting proving grounds for the development of biologically inspired hardware and software—machines that try to recreate billions of years of evolution with some of the abilities and characteristics of living beings. Humanoids could be especially useful for their ability to “live” in human-populated environments, occupying the same physical space as people and using tools that have been designed for people. Natural human–robot interaction is also an important facet of humanoid research. Finally, learning and adapting from experience, the hallmark of human intelligence, may require some approximation to the human body in order to attain similar capacities to humans. This chapter focuses particularly on compliant actuation, soft robotics, biomimetic robot vision, robot touch, and brain-inspired motor control in the context of the iCub humanoid robot.

scholarly journals Moment-to-Moment Continuous Attention Fluctuation Monitoring through Consumer-Grade EEG Device

Sensors ◽  
2021 ◽  
Vol 21 (10) ◽  
pp. 3419
Author(s):  
Shan Zhang ◽  
Zihan Yan ◽  
Shardul Sapkota ◽  
Shengdong Zhao ◽  
Wei Tsang Ooi
Keyword(s):  
Ground Truth ◽  
Detection Accuracy ◽  
Continuous Performance Task ◽  
Eeg Signals ◽  
Gradual Onset ◽  
Machine Learning Approach ◽  
A New Technique ◽  
Thought Probes ◽  
Labeling Method ◽  
Fluctuation Measurement

While numerous studies have explored using various sensing techniques to measure attention states, moment-to-moment attention fluctuation measurement is unavailable. To bridge this gap, we applied a novel paradigm in psychology, the gradual-onset continuous performance task (gradCPT), to collect the ground truth of attention states. GradCPT allows for the precise labeling of attention fluctuation on an 800 ms time scale. We then developed a new technique for measuring continuous attention fluctuation, based on a machine learning approach that uses the spectral properties of EEG signals as the main features. We demonstrated that, even using a consumer grade EEG device, the detection accuracy of moment-to-moment attention fluctuations was 73.49%. Next, we empirically validated our technique in a video learning scenario and found that our technique match with the classification obtained through thought probes, with an average F1 score of 0.77. Our results suggest the effectiveness of using gradCPT as a ground truth labeling method and the feasibility of using consumer-grade EEG devices for continuous attention fluctuation detection.

scholarly journals How does the robot feel? Perception of valence and arousal in emotional body language

2018 ◽  
Vol 9 (1) ◽  
pp. 168-182 ◽  
Author(s):  
Mina Marmpena ◽  
Angelica Lim ◽  
Torbjørn S. Dahl
Keyword(s):  
Human Perception ◽  
Ground Truth ◽  
Body Language ◽  
Human Robot Interaction ◽  
Behavioral Experiments ◽  
Robot Interaction ◽  
Expression Of Emotion ◽  
Exploratory Approach ◽  
Starting Point ◽  
Valence And Arousal

Abstract Human-robot interaction in social robotics applications could be greatly enhanced by robotic behaviors that incorporate emotional body language. Using as our starting point a set of pre-designed, emotion conveying animations that have been created by professional animators for the Pepper robot, we seek to explore how humans perceive their affect content, and to increase their usability by annotating them with reliable labels of valence and arousal, in a continuous interval space. We conducted an experiment with 20 participants who were presented with the animations and rated them in the two-dimensional affect space. An inter-rater reliability analysis was applied to support the aggregation of the ratings for deriving the final labels. The set of emotional body language animations with the labels of valence and arousal is available and can potentially be useful to other researchers as a ground truth for behavioral experiments on robotic expression of emotion, or for the automatic selection of robotic emotional behaviors with respect to valence and arousal. To further utilize the data we collected, we analyzed it with an exploratory approach and we present some interesting trends with regard to the human perception of Pepper’s emotional body language, that might be worth further investigation.

scholarly journals A Taxonomy in Robot-Assisted Training: Current Trends, Needs and Challenges

Technologies ◽  
2018 ◽  
Vol 6 (4) ◽  
pp. 119 ◽  
Author(s):  
Konstantinos Tsiakas ◽  
Maria Kyrarini ◽  
Vangelis Karkaletsis ◽  
Fillia Makedon ◽  
Oliver Korn
Keyword(s):  
Training Session ◽  
Training System ◽  
Human Robot Interaction ◽  
Design Development ◽  
Robot Assisted ◽  
Robot Interaction ◽  
Training Systems ◽  
Robot Perception ◽  
Educational Environments ◽  
And Behavior

In this article, we present a taxonomy in Robot-Assisted Training; a growing body of research in Human–Robot Interaction which focuses on how robotic agents and devices can be used to enhance user’s performance during a cognitive or physical training task. Robot-Assisted Training systems have been successfully deployed to enhance the effects of a training session in various contexts, i.e., rehabilitation systems, educational environments, vocational settings, etc. The proposed taxonomy suggests a set of categories and parameters that can be used to characterize such systems, considering the current research trends and needs for the design, development and evaluation of Robot-Assisted Training systems. To this end, we review recent works and applications in Robot-Assisted Training systems, as well as related taxonomies in Human–Robot Interaction. The goal is to identify and discuss open challenges, highlighting the different aspects of a Robot-Assisted Training system, considering both robot perception and behavior control.

scholarly journals A Bio-inspired Motivational Decision Making System for Social Robots Based on the Perception of the User

Sensors ◽  
2018 ◽  
Vol 18 (8) ◽  
pp. 2691 ◽  
Author(s):  
Marcos Maroto-Gómez ◽  
Álvaro Castro-González ◽  
José Castillo ◽  
María Malfaz ◽  
Miguel Salichs
Keyword(s):  
Decision Making ◽  
Human Robot Interaction ◽  
Learning Mechanisms ◽  
Biologically Inspired ◽  
Robot Interaction ◽  
Detection Mechanism ◽  
Learned Behaviors ◽  
Decision Making System ◽  
Real Robot ◽  
Self Learning

Nowadays, many robotic applications require robots making their own decisions and adapting to different conditions and users. This work presents a biologically inspired decision making system, based on drives, motivations, wellbeing, and self-learning, that governs the behavior of the robot considering both internal and external circumstances. In this paper we state the biological foundations that drove the design of the system, as well as how it has been implemented in a real robot. Following a homeostatic approach, the ultimate goal of the robot is to keep its wellbeing as high as possible. In order to achieve this goal, our decision making system uses learning mechanisms to assess the best action to execute at any moment. Considering that the proposed system has been implemented in a real social robot, human-robot interaction is of paramount importance and the learned behaviors of the robot are oriented to foster the interactions with the user. The operation of the system is shown in a scenario where the robot Mini plays games with a user. In this context, we have included a robust user detection mechanism tailored for short distance interactions. After the learning phase, the robot has learned how to lead the user to interact with it in a natural way.

scholarly journals Movement Estimation Using Soft Sensors Based on Bi-LSTM and Two-Layer LSTM for Human Motion Capture

Sensors ◽  
2020 ◽  
Vol 20 (6) ◽  
pp. 1801 ◽  
Author(s):  
Haitao Guo ◽  
Yunsick Sung
Keyword(s):  
Motion Capture ◽  
Short Term Memory ◽  
Arm Movements ◽  
Ground Truth ◽  
Human Movement ◽  
Human Motion ◽  
Soft Sensors ◽  
Upper Arm ◽  
Human Motion Capture ◽  
Gesture Control

The importance of estimating human movement has increased in the field of human motion capture. HTC VIVE is a popular device that provides a convenient way of capturing human motions using several sensors. Recently, the motion of only users’ hands has been captured, thereby greatly reducing the range of motion captured. This paper proposes a framework to estimate single-arm orientations using soft sensors mainly by combining a Bi-long short-term memory (Bi-LSTM) and two-layer LSTM. Positions of the two hands are measured using an HTC VIVE set, and the orientations of a single arm, including its corresponding upper arm and forearm, are estimated using the proposed framework based on the estimated positions of the two hands. Given that the proposed framework is meant for a single arm, if orientations of two arms are required to be estimated, the estimations are performed twice. To obtain the ground truth of the orientations of single-arm movements, two Myo gesture-control sensory armbands are employed on the single arm: one for the upper arm and the other for the forearm. The proposed framework analyzed the contextual features of consecutive sensory arm movements, which provides an efficient way to improve the accuracy of arm movement estimation. In comparison with the ground truth, the proposed method estimated the arm movements using a dynamic time warping distance, which was the average of 73.90% less than that of a conventional Bayesian framework. The distinct feature of our proposed framework is that the number of sensors attached to end-users is reduced. Additionally, with the use of our framework, the arm orientations can be estimated with any soft sensor, and good accuracy of the estimations can be ensured. Another contribution is the suggestion of the combination of the Bi-LSTM and two-layer LSTM.

scholarly journals Using Wearable Sensors and Machine Learning to Automatically Detect Freezing of Gait during a FOG-Provoking Test

Sensors ◽  
2020 ◽  
Vol 20 (16) ◽  
pp. 4474 ◽  
Author(s):  
Tal Reches ◽  
Moria Dagan ◽  
Talia Herman ◽  
Eran Gazit ◽  
Natalia A. Gouskova ◽  
...  
Keyword(s):  
Machine Learning ◽  
Wearable Sensors ◽  
Freezing Of Gait ◽  
Ground Truth ◽  
Self Report ◽  
Therapeutic Interventions ◽  
Percent Time ◽  
Machine Learning Approach ◽  
Automated Machine Learning ◽  
Fog Detection

Freezing of gait (FOG) is a debilitating motor phenomenon that is common among individuals with advanced Parkinson’s disease. Objective and sensitive measures are needed to better quantify FOG. The present work addresses this need by leveraging wearable devices and machine-learning methods to develop and evaluate automated detection of FOG and quantification of its severity. Seventy-one subjects with FOG completed a FOG-provoking test while wearing three wearable sensors (lower back and each ankle). Subjects were videotaped before (OFF state) and after (ON state) they took their antiparkinsonian medications. Annotations of the videos provided the “ground-truth” for FOG detection. A leave-one-patient-out validation process with a training set of 57 subjects resulted in 84.1% sensitivity, 83.4% specificity, and 85.0% accuracy for FOG detection. Similar results were seen in an independent test set (data from 14 other subjects). Two derived outcomes, percent time frozen and number of FOG episodes, were associated with self-report of FOG. Bother derived-metrics were higher in the OFF state than in the ON state and in the most challenging level of the FOG-provoking test, compared to the least challenging level. These results suggest that this automated machine-learning approach can objectively assess FOG and that its outcomes are responsive to therapeutic interventions.

Real-time Auditory and Visual Multiple-speaker Tracking For Human-robot Interaction

2002 ◽  
Vol 14 (5) ◽  
pp. 479-489 ◽  
Author(s):  
Kazuhiro Nakadai ◽  
◽  
Ken-ichi Hidai ◽  
Hiroshi G. Okuno ◽  
Hiroshi Mizoguchi ◽  
...  
Keyword(s):  
Social Interaction ◽  
Real Time ◽  
Human Robot Interaction ◽  
Intensity Difference ◽  
Real Time Processing ◽  
Time Processing ◽  
Robot Perception ◽  
Interaural Phase ◽  
Speaker Tracking ◽  
Interaural Phase Difference

This paper addresses real-time multiple speaker tracking because it is essential in robot perception and human-robot social interaction. The difficulty lies in treating a mixture of sounds, occlusion (some speakers are hidden) and real-time processing. Our approach consists of three components: (1) the extraction of the direction of each speaker by using interaural phase difference and interaural intensity difference, (2) the resolution of each speakers direction by multimodal integration of audition, vision and motion with canceling inevitable motor noises in motion in case of an unseen or silent speaker, and (3) the distributed implementation to three PCs connected by TCP/IP network to attain real-time processing. As a result, we attain robust real-time speaker tracking with 200 ms delay in a non-anechoic room, even when multiple speakers exist and the tracking person is visually occluded. In addition, the feasibility of social interaction is shown through application of our technique to a receptionist robot and a companion robot at a party.

scholarly journals An Immersive Investment Game to Study Human-Robot Trust

2021 ◽  
Vol 8 ◽  
Author(s):  
Sebastian Zörner ◽  
Emy Arts ◽  
Brenda Vasiljevic ◽  
Ankit Srivastava ◽  
Florian Schmalzl ◽  
...  
Keyword(s):  
Science Fiction ◽  
Positive Impact ◽  
Space Mission ◽  
Verbal Communication ◽  
Human Robot Interaction ◽  
Robot Interaction ◽  
Investment Game ◽  
Robot Perception ◽  
The Impact ◽  
Human Robot Collaboration

As robots become more advanced and capable, developing trust is an important factor of human-robot interaction and cooperation. However, as multiple environmental and social factors can influence trust, it is important to develop more elaborate scenarios and methods to measure human-robot trust. A widely used measurement of trust in social science is the investment game. In this study, we propose a scaled-up, immersive, science fiction Human-Robot Interaction (HRI) scenario for intrinsic motivation on human-robot collaboration, built upon the investment game and aimed at adapting the investment game for human-robot trust. For this purpose, we utilize two Neuro-Inspired COmpanion (NICO) - robots and a projected scenery. We investigate the applicability of our space mission experiment design to measure trust and the impact of non-verbal communication. We observe a correlation of 0.43 (p=0.02) between self-assessed trust and trust measured from the game, and a positive impact of non-verbal communication on trust (p=0.0008) and robot perception for anthropomorphism (p=0.007) and animacy (p=0.00002). We conclude that our scenario is an appropriate method to measure trust in human-robot interaction and also to study how non-verbal communication influences a human’s trust in robots.

Sign in / Sign up

Export Citation Format

Share Document