scholarly journals Hierarchical generative modelling for autonomous robots

2021 ◽  
Author(s):  
Kai Yuan ◽  
Noor Sajid ◽  
Karl Friston ◽  
Zhibin Li
Keyword(s):  
Motor Control ◽  
Humanoid Robot ◽  
Autonomous Robots ◽  
Generative Model ◽  
Hierarchical Architecture ◽  
Fundamental Aspect ◽  
Task Completion ◽  
Final Destination ◽  
High Level ◽  
Generative Modelling

Abstract Humans can produce complex movements when interacting with their surroundings. This relies on the planning of various movements and subsequent execution. In this paper, we investigated this fundamental aspect of motor control in the setting of autonomous robotic operations. We consider hierarchical generative modelling—for autonomous task completion—that mimics the deep temporal architecture of human motor control. Here, temporal depth refers to the nested time scales at which successive levels of a forward or generative model unfold: for example, the apprehension and delivery of an object requires both a global plan that contextualises the fast coordination of multiple local limb movements. This separation of temporal scales can also be motivated from a robotics and control perspective. Specifically, to ensure versatile sensorimotor control, it is necessary to hierarchically structure high-level planning and low-level motor control of individual limbs. We use numerical experiments to establish the efficacy of this formulation and demonstrate how a humanoid robot can autonomously solve a complex task requiring locomotion, manipulation, and grasping, using a hierarchical generative model. In particular, the humanoid robot can retrieve and deliver a box, open and walk through a door to reach the final destination. Our approach, and experiments, illustrate the effectiveness of using human-inspired motor control algorithms, which provide a scalable hierarchical architecture for autonomous performance of complex goal-directed tasks.

The Pransky interview: Brennard Pierce: world class robotics designer and serial entrepreneur

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Joanne Pransky
Keyword(s):  
Humanoid Robot ◽  
Autonomous Robots ◽  
Industrial Robot ◽  
Biped Robot ◽  
Interdisciplinary Teams ◽  
Mobile Platforms ◽  
Content Type ◽  
Serial Entrepreneur ◽  
World Class ◽  
High Level

Purpose The purpose of this paper is to provide a “Q&A interview” conducted by Joanne Pransky of Industrial Robot Journal as a method to impart the combined technological, business and personal experience of a prominent, robotic industry engineer-turned entrepreneur regarding his pioneering efforts in starting robotic companies and commercializing technological inventions. The paper aims to discuss these issues. Design/methodology/approach The interviewee is Brennard Pierce, a world-class robotics designer and serial entrepreneur. Pierce is currently consulting in robotics after exiting from his latest startup as cofounder and chief robotics officer of Bear Robotics. Pierce discusses what led him to the field of robotics, the success of Bear Robotics, the challenges he’s faced and his future goals. Findings Pierce received a Bachelor of Science in computer science from Exeter University. He then founded his first startup, 5TWO, a custom software company. Always passionate about robotics as a hobby and now wanting to pursue the field professionally, he sold 5TWO to obtain a Master of Science, Robotics degree from the newly formed Bristol Robotics Lab (BRL) at Bristol University. After BRL, where he designed and built a biped robot that learned to walk using evolutionary algorithms, he joined the Robotics Research team at Carnegie Mellon University where he worked on a full-size humanoid robot for a large electronics company, designing and executing simple experiments for balancing. He then spent the next six years as a PhD candidate and robotics researcher at the Technical University Munich (TUM), Institute for Cognitive Science, where he built a compliant humanoid robot and a new generation of field programmable gate array-based robotic controllers. Afterwards, Pierce established the robotic startup Robotise in Munich to commercialize the omni-directional mobile platforms that he had developed at TUM. A couple of years later, Pierce left Robotise to cofound Bear Robotics, a Silicon Valley based company that brings autonomous robots to the restaurant industry. He remained at Bear Robotics for four years as chief robotics officer. He is presently a robotics consultant, waiting for post-COVID before beginning his next robotic startup. Originality/value Pierce is a seasoned roboticist and a successful entrepreneur. He has 15+ years’ of unique experience in both designing robotic hardware and writing low level embedded and high level cloud software. During his career he has founded three companies, managed small to middle sized interdisciplinary teams, and hired approximately 100 employees of all levels. Pierce’s robotic startup in Munich, Robotise, was solely based on his idea, design and implementation for an autonomous mobile delivery system. The third company he cofounded, Bear Robotics, successfully raised a $32m Series A funding lead by SoftBank. Bear Robotics is the recipient of the USA’s National Restaurant Association Kitchen Innovation Award; Fast Company’s World Changing Ideas Awards; and the Hospitality Innovation Planet 2020 Award.

scholarly journals Improved Mutual Understanding for Human-Robot Collaboration: Combining Human-Aware Motion Planning with Haptic Feedback Devices for Communicating Planned Trajectory

Sensors ◽  
2021 ◽  
Vol 21 (11) ◽  
pp. 3673
Author(s):  
Stefan Grushko ◽  
Aleš Vysocký ◽  
Petr Oščádal ◽  
Michal Vocetka ◽  
Petr Novák ◽  
...  
Keyword(s):  
User Study ◽  
Haptic Feedback ◽  
Fundamental Aspect ◽  
Task Completion ◽  
Good Efficiency ◽  
Notification System ◽  
Movement Data ◽  
Novel Approach ◽  
Machine Interface ◽  
High Degree

In a collaborative scenario, the communication between humans and robots is a fundamental aspect to achieve good efficiency and ergonomics in the task execution. A lot of research has been made related to enabling a robot system to understand and predict human behaviour, allowing the robot to adapt its motion to avoid collisions with human workers. Assuming the production task has a high degree of variability, the robot’s movements can be difficult to predict, leading to a feeling of anxiety in the worker when the robot changes its trajectory and approaches since the worker has no information about the planned movement of the robot. Additionally, without information about the robot’s movement, the human worker cannot effectively plan own activity without forcing the robot to constantly replan its movement. We propose a novel approach to communicating the robot’s intentions to a human worker. The improvement to the collaboration is presented by introducing haptic feedback devices, whose task is to notify the human worker about the currently planned robot’s trajectory and changes in its status. In order to verify the effectiveness of the developed human-machine interface in the conditions of a shared collaborative workspace, a user study was designed and conducted among 16 participants, whose objective was to accurately recognise the goal position of the robot during its movement. Data collected during the experiment included both objective and subjective parameters. Statistically significant results of the experiment indicated that all the participants could improve their task completion time by over 45% and generally were more subjectively satisfied when completing the task with equipped haptic feedback devices. The results also suggest the usefulness of the developed notification system since it improved users’ awareness about the motion plan of the robot.

scholarly journals Infants in Control: Prospective Motor Control and Executive Functions in Action Development

2018 ◽  
Author(s):  
Janna M. Gottwald
Keyword(s):  
Motor Control ◽  
Cognitive Control ◽  
Executive Functions ◽  
Visual Information ◽  
Motion Tracking ◽  
Planning Process ◽  
Action Planning ◽  
Sources Of Information ◽  
Action Sequences ◽  
High Level

This thesis assesses the link between action and cognition early in development. Thus the notion of an embodied cognition is investigated by tying together two levels of action control in the context of reaching in infancy: prospective motor control and executive functions. The ability to plan our actions is the inevitable foundation of reaching our goals. Thus actions can be stratified on different levels of control. There is the relatively low level of prospective motor control and the comparatively high level of cognitive control. Prospective motor control is concerned with goal-directed actions on the level of single movements and movement combinations of our body and ensures purposeful, coordinated movements, such as reaching for a cup of coffee. Cognitive control, in the context of this thesis more precisely referred to as executive functions, deals with goal-directed actions on the level of whole actions and action combinations and facilitates directedness towards mid- and long-term goals, such as finishing a doctoral thesis. Whereas prospective motor control and executive functions are well studied in adulthood, the early development of both is not sufficiently understood.This thesis comprises three empirical motion-tracking studies that shed light on prospective motor control and executive functions in infancy. Study I investigated the prospective motor control of current actions by having 14-month-olds lift objects of varying weights. In doing so, multi-cue integration was addressed by comparing the use of visual and non-visual information to non-visual information only. Study II examined the prospective motor control of future actions in action sequences by investigating reach-to-place actions in 14-month-olds. Thus the extent to which Fitts’ law can explain movement duration in infancy was addressed. Study III lifted prospective motor control to a higher that is cognitive level, by investigating it relative to executive functions in 18-months-olds.Main results were that 14-month-olds are able to prospectively control their manual actions based on object weight. In this action planning process, infants use different sources of information. Beyond this ability to prospectively control their current action, 14-month-olds also take future actions into account and plan their actions based on the difficulty of the subsequentaction in action sequences. In 18-month-olds, prospective motor control in manual actions, such as reaching, is related to early executive functions, as demonstrated for behavioral prohibition and working memory. These findings are consistent with the idea that executive functions derive from prospective motor control. I suggest that executive functions could be grounded in the development of motor control. In other words, early executive functions should be seen as embodied.

scholarly journals Sequence Braiding: Visual Overviews of Temporal Event Sequences and Attributes

2020 ◽  
Author(s):  
Sara Di Bartolomeo ◽  
Yixuan Zhang ◽  
Fangfang Sheng ◽  
Cody Dunne
Keyword(s):  
Completion Time ◽  
Source Code ◽  
Good Evidence ◽  
Task Completion ◽  
Event Sequences ◽  
Sentinel Events ◽  
Temporal Events ◽  
High Level ◽  
Over Time ◽  
Temporal Event Sequences

Temporal event sequence alignment has been used in many domains to visualize nuanced changes and interactions over time. Existing approaches align one or two sentinel events. Overview tasks require examining all alignments of interest using interaction and time or juxtaposition of many visualizations. Furthermore, any event attribute overviews are not closely tied to sequence visualizations. We present SEQUENCE BRAIDING, a novel overview visualization for temporal event sequences and attributes using a layered directed acyclic network.SEQUENCE BRAIDING visually aligns many temporal events and attribute groups simultaneously and supports arbitrary ordering, absence, and duplication of events. In a controlled experiment we compare SEQUENCE BRAIDING and IDMVis on user task completion time, correctness, error, and confidence. Our results provide good evidence that users of SEQUENCE BRAIDING can understand high-level patterns and trends faster and with similar error. A full version of this paper with all appendices;the evaluation stimuli, data, and analysis code; and source code are available at osf.io/s92bu.

scholarly journals Developing a protocol and experimental setup for using a humanoid robot to assist children with autism to develop visual perspective taking skills

2019 ◽  
Vol 10 (1) ◽  
pp. 167-179 ◽  
Author(s):  
Luke Jai Wood ◽  
Ben Robins ◽  
Gabriella Lakatos ◽  
Dag Sverre Syrdal ◽  
Abolfazl Zaraki ◽  
...  
Keyword(s):  
Perspective Taking ◽  
Humanoid Robot ◽  
Children With Autism ◽  
Fundamental Aspect ◽  
Visual Perspective ◽  
Inclusion Criteria ◽  
Future Studies ◽  
Visual Perspective Taking ◽  
The World

AbstractVisual Perspective Taking (VPT) is the ability to see the world from another person’s perspective, taking into account what they see and how they see it, drawing upon both spatial and social information. Children with autism often find it difficult to understand that other people might have perspectives, viewpoints, beliefs and knowledge that are different from their own, which is a fundamental aspect of VPT. In this research we aimed to develop a methodology to assist children with autism develop their VPT skills using a humanoid robot and present results from our first long-term pilot study. The games we devised were implemented with the Kaspar robot and, to our knowledge, this is the first attempt to improve the VPT skills of children with autism through playing and interacting with a humanoid robot.We describe in detail the standard pre- and post-assessments that we performed with the children in order to measure their progress and also the inclusion criteria derived fromthe results for future studies in this field. Our findings suggest that some children may benefit from this approach of learning about VPT, which shows that this approach merits further investigation.

On the Problem of Moving Objects With Autonomous Robots: A Unifying High-Level Planning Approach

2016 ◽  
Vol 1 (1) ◽  
pp. 469-476 ◽  
Author(s):  
Hamal Marino ◽  
Mirko Ferrati ◽  
Alessandro Settimi ◽  
Carlos Rosales ◽  
Marco Gabiccini
Keyword(s):  
Moving Objects ◽  
Autonomous Robots ◽  
Planning Approach ◽  
High Level

scholarly journals Automated movement assessment in stroke rehabilitation

2021 ◽  
Author(s):  
Tamim Ahmed ◽  
Kowshik Thopalli ◽  
Thanassis Rikakis ◽  
Pavan Turaga ◽  
Aisling Kelliher ◽  
...  
Keyword(s):  
Upper Extremity ◽  
Hierarchical Model ◽  
Stroke Rehabilitation ◽  
Expert Knowledge ◽  
Low Cost ◽  
Data Driven ◽  
Task Completion ◽  
Convolutional Network ◽  
Rgb Images ◽  
High Level

We are developing a system for long-term Semi-Automated Rehabilitation At the Home (SARAH) that relies on low-cost and unobtrusive video-based sensing. We present a cyber-human methodology used by the SARAH system for automated assessment of upper extremity stroke rehabilitation at the home. We propose a hierarchical model for automatically segmenting stroke survivor's movements and generating training task performance assessment scores during rehabilitation. The hierarchical model fuses expert therapist knowledge-based approaches with data-driven techniques. The expert knowledge is more observable in the higher layers of the hierarchy (task and segment) and therefore more accessible to algorithms incorporating high-level constraints relating to activity structure (i.e. type and order of segments per task). We utilize an HMM and a Decision Tree model to connect these high-level priors to data-driven analysis. The lower layers (RGB images and raw kinematics) need to be addressed primarily through data-driven techniques. We use a transformer-based architecture operating on low-level action features (tracking of individual body joints and objects) and a Multi-Stage Temporal Convolutional Network(MS-TCN) operating on raw RGB images. We develop a sequence combining these complementary algorithms effectively, thus encoding the information from different layers of the movement hierarchy. Through this combination, we produce robust segmentation and task assessment results on noisy, variable, and limited data, which is characteristic of low-cost video capture of rehabilitation at the home. Our proposed approach achieves 85% accuracy in per-frame labeling, 99% accuracy in segment classification, and 93% accuracy in task completion assessment. Although the methodology proposed in this paper applies to upper extremity rehabilitation using the SARAH system, it can potentially be used, with minor alterations, to assist automation in many other movement rehabilitation contexts (i.e. lower extremity training for neurological accidents).

Sign in / Sign up

Export Citation Format

Share Document