From Visual Understanding to Complex Object Manipulation

2019 ◽  
Vol 2 (1) ◽  
pp. 161-179 ◽  
Author(s):  
Judith Bütepage ◽  
Silvia Cruciani ◽  
Mia Kokic ◽  
Michael Welle ◽  
Danica Kragic
Keyword(s):  
Temporal Integration ◽  
Object Manipulation ◽  
Robotic Systems ◽  
Sensorimotor Learning ◽  
Complex Object ◽  
Task Constraints ◽  
Grasp Planning ◽  
Temporal Aspects ◽  
Object Use ◽  
Visual Understanding

Planning and executing object manipulation requires integrating multiple sensory and motor channels while acting under uncertainty and complying with task constraints. As the modern environment is tuned for human hands, designing robotic systems with similar manipulative capabilities is crucial. Research on robotic object manipulation is divided into smaller communities interested in, e.g., motion planning, grasp planning, sensorimotor learning, and tool use. However, few attempts have been made to combine these areas into holistic systems. In this review, we aim to unify the underlying mechanics of grasping and in-hand manipulation by focusing on the temporal aspects of manipulation, including visual perception, grasp planning and execution, and goal-directed manipulation. Inspired by human manipulation, we envision that an emphasis on the temporal integration of these processes opens the way for human-like object use by robots.

Hand closure model for planning top grasps with soft robotic hands

2020 ◽  
Vol 39 (14) ◽  
pp. 1706-1723
Author(s):  
Maria Pozzi ◽  
Sara Marullo ◽  
Gionata Salvietti ◽  
Joao Bimbo ◽  
Monica Malvezzi ◽  
...  
Keyword(s):  
Robotic Systems ◽  
Closure Model ◽  
Robotic Hands ◽  
Grasp Planning ◽  
Model Based ◽  
Major Trend ◽  
Different Shapes ◽  
Planning Strategies ◽  
Experimental Trials ◽  
Soft Manipulation

Automating the act of grasping is one of the most compelling challenges in robotics. In recent times, a major trend has gained the attention of the robotic grasping community: soft manipulation. Along with the design of intrinsically soft robotic hands, it is important to devise grasp planning strategies that can take into account the hand characteristics, but are general enough to be applied to different robotic systems. In this article, we investigate how to perform top grasps with soft hands according to a model-based approach, using both power and precision grasps. The so-called closure signature (CS) is used to model closure motions of soft hands by associating to them a preferred grasping direction. This direction can be aligned to a suitable direction over the object to achieve successful top grasps. The CS-alignment is here combined with a recently developed AI-driven grasp planner for rigid grippers that is adjusted and used to retrieve an estimate of the optimal grasp to be performed on the object. The resulting grasp planner is tested with multiple experimental trials with two different robotic hands. A wide set of objects with different shapes was grasped successfully.

Grip force anticipation of nonlinear, underactuated load force

2021 ◽  
Author(s):  
Francis M. Grover ◽  
Christopher Riehm ◽  
Paula L. Silva ◽  
Tamara Lorenz ◽  
Michael A. Riley
Keyword(s):  
Force Control ◽  
Internal Model ◽  
Grip Force ◽  
Object Manipulation ◽  
Load Force ◽  
Complex Object ◽  
Active Object ◽  
Grip Force Control ◽  
Motor Commands ◽  
Model Approach

Feedforward internal model-based control enabled by efference copies of motor commands is the prevailing theoretical account of motor anticipation. Grip force control during object manipulation-a paradigmatic example of motor anticipation-is a key line of evidence for that account. However, the internal model approach has not addressed the computational challenges faced by the act of manipulating mechanically complex objects with nonlinear, underactuated degrees of freedom. These objects exhibit complex and unpredictable load force dynamics which cannot be encoded by efference copies of underlying motor commands, leading to the prediction from the perspective of an efference copy-enabled feedforward control scheme that grip force should either lag or fail to coordinate with changes in load force. In contrast to that prediction, we found evidence for strong, precise, anticipatory grip force control during manipulations of a complex object. The results are therefore inconsistent with the internal forward model approach and suggest that efference copies of motor commands are not necessary to enable anticipatory control during active object manipulation.

scholarly journals Energy margins in dynamic object manipulation

2012 ◽  
Vol 108 (5) ◽  
pp. 1349-1365 ◽  
Author(s):  
Christopher J. Hasson ◽  
Tian Shen ◽  
Dagmar Sternad
Keyword(s):  
Safety Margin ◽  
Object Manipulation ◽  
Minimum Time ◽  
Complex Objects ◽  
Task Constraints ◽  
Target Time ◽  
Time Task ◽  
Safety Margins ◽  
Movement Strategies ◽  
Single Strategy

Many tasks require humans to manipulate dynamically complex objects and maintain appropriate safety margins, such as placing a cup of coffee on a coaster without spilling. This study examined how humans learn such safety margins and how they are shaped by task constraints and changing variability with improved skill. Eighteen subjects used a manipulandum to transport a shallow virtual cup containing a ball to a target without losing the ball. Half were to complete the cup transit in a comfortable target time of 2 s (a redundant task with infinitely many equivalent solutions), and the other half in minimum time (a nonredundant task with one explicit cost to optimize). The safety margin was defined as the ball energy relative to escape, i.e., as an energy margin. The first hypothesis, that subjects converge to a single strategy in the minimum-time task but choose different strategies in the less constrained target-time task, was not supported. Both groups developed individualized strategies with practice. The second hypothesis, that subjects decrease safety margins in the minimum-time task but increase them in the target-time task, was supported. The third hypothesis, that in both tasks subjects modulate energy margins according to their execution variability, was partially supported. In the target-time group, changes in energy margins correlated positively with changes in execution variability; in the minimum-time group, such a relation was observed only at the end of practice, not across practice. These results show that when learning a redundant object manipulation task, most subjects increase their safety margins and shape their movement strategies in accordance with their changing variability.

scholarly journals Emotional arousal ripples across time to bind subsequent episodes in memory

2021 ◽  
Author(s):  
David Clewett ◽  
Lila Davachi
Keyword(s):  
Temporal Order ◽  
Temporal Integration ◽  
Boundary Spanning ◽  
Emotional States ◽  
Ample Evidence ◽  
Auditory Event ◽  
Event Sequences ◽  
Negative Arousal ◽  
Temporal Aspects ◽  
Temporal Order Memory

Time unfolds continuously, yet our memories are stored as discrete episodes. Prior work shows that fluctuations between stability and change in an ongoing neutral context facilitates this formation of distinct and memorable events. However, less is known about how shifting emotional states influence these memory processes, despite ample evidence that emotion has a robust influence on non-temporal aspects of episodic memory. Here, we examined if emotional stimuli influence temporal memory for recent event sequences. Participants encoded lists of neutral object images while listening to pure auditory tones. At regular intervals within each list, participants heard emotional positive, negative, or neutral sounds, which served as ‘emotional event boundaries’ that divided each sequence into discrete auditory events. Temporal order memory was tested for neutral item pairs that either spanned an emotional sound (‘boundary-spanning’) or encountered within the same auditory event (‘same-context’). We found that highly arousing boundaries had opposite effects on binding ongoing versus subsequent sequential representations in memory. Specifically, highly arousing emotional sounds tended to lead to worse temporal order memory for boundary-spanning item pairs. By contrast, they led to better temporal order memory for same-context item pairs in the next event. Both of these arousal effects were specific to negative sounds. The carryover effect of negative arousal was also strongest for item pairs encountered closest to the boundary and diminished as the event unfolded. These findings suggest that temporally dynamic emotional states support the temporal integration of mnemonic events, which may contribute to the hyper-episodic nature of negative emotional memories.

scholarly journals Temporal Integration of Text Transcripts and Acoustic Features for Alzheimer's Diagnosis Based on Spontaneous Speech

2021 ◽  
Vol 13 ◽  
Author(s):  
Matej Martinc ◽  
Fasih Haider ◽  
Senja Pollak ◽  
Saturnino Luz
Keyword(s):  
Machine Learning ◽  
Structural Information ◽  
Temporal Integration ◽  
Spontaneous Speech ◽  
Data Representation ◽  
Acoustic Features ◽  
Word Level ◽  
Temporal Aspects ◽  
Textual Features ◽  
And Gender

Background: Advances in machine learning (ML) technology have opened new avenues for detection and monitoring of cognitive decline. In this study, a multimodal approach to Alzheimer's dementia detection based on the patient's spontaneous speech is presented. This approach was tested on a standard, publicly available Alzheimer's speech dataset for comparability. The data comprise voice samples from 156 participants (1:1 ratio of Alzheimer's to control), matched by age and gender.Materials and Methods: A recently developed Active Data Representation (ADR) technique for voice processing was employed as a framework for fusion of acoustic and textual features at sentence and word level. Temporal aspects of textual features were investigated in conjunction with acoustic features in order to shed light on the temporal interplay between paralinguistic (acoustic) and linguistic (textual) aspects of Alzheimer's speech. Combinations between several configurations of ADR features and more traditional bag-of-n-grams approaches were used in an ensemble of classifiers built and evaluated on a standardised dataset containing recorded speech of scene descriptions and textual transcripts.Results: Employing only semantic bag-of-n-grams features, an accuracy of 89.58% was achieved in distinguishing between Alzheimer's patients and healthy controls. Adding temporal and structural information by combining bag-of-n-grams features with ADR audio/textual features, the accuracy could be improved to 91.67% on the test set. An accuracy of 93.75% was achieved through late fusion of the three best feature configurations, which corresponds to a 4.7% improvement over the best result reported in the literature for this dataset.Conclusion: The proposed combination of ADR audio and textual features is capable of successfully modelling temporal aspects of the data. The machine learning approach toward dementia detection achieves best performance when ADR features are combined with strong semantic bag-of-n-grams features. This combination leads to state-of-the-art performance on the AD classification task.

scholarly journals Eliciting End-State Comfort Planning in Children With and Without Developmental Coordination Disorder Using a Hammer Task: A Pilot Study

2021 ◽  
Vol 12 ◽  
Author(s):  
Hilde Krajenbrink ◽  
Jessica Mireille Lust ◽  
Bert Steenbergen
Keyword(s):  
Pilot Study ◽  
Developmental Coordination Disorder ◽  
Healthy Adults ◽  
Complex Object ◽  
Task Constraints ◽  
Planning Strategy ◽  
Wooden Block ◽  
Coordination Disorder ◽  
Present Pilot Study ◽  
Consistent Tendency

The end-state comfort (ESC) effect refers to the consistent tendency of healthy adults to end their movements in a comfortable end posture. In children with and without developmental coordination disorder (DCD), the results of studies focusing on ESC planning have been inconclusive, which is likely to be due to differences in task constraints. The present pilot study focused on the question whether children with and without DCD were able to change their planning strategy and were more likely to plan for ESC when demanded by complex object manipulations at the end of a task. To this end, we examined ESC planning in 18 children with and without DCD (aged 5–11years) using the previously used sword-task and the newly developed hammer-task. In the sword-task, children had to insert a sword in a wooden block, which could be relatively easily completed with an uncomfortable end-posture. In the hammer-task, children had to strike down a nail in a wooden pounding bench, which required additional force and speed demands, making it relatively difficult to complete the movement with an uncomfortable end-posture. In line with our hypothesis, the results demonstrated that children with and without DCD were more likely to plan for ESC on the hammer-task compared with the sword-task. Thus, while children with and without DCD show inconsistent ESC planning on many previously used tasks, the present pilot study shows that many of them are able to take into account the end-state of their movements if demanded by task constraints.

Sign in / Sign up

Export Citation Format

Share Document