Mix Frame Visual Servo Control Framework for Autonomous Assistive Robotic Arms

Assistive robotic arms (ARAs) that provide care to the elderly and people with disabilities, are a significant part of Human-Robot Interaction (HRI). Presently available ARAs provide non-intuitive interfaces such as joysticks for control and thus, lacks the autonomy to perform daily activities. This study proposes that, for inducing autonomous behavior in ARAs, visual sensors integration is vital, and visual servoing in the direct Cartesian control mode is the preferred method. Generally, ARAs are designed in a configuration where its end-effector’s position is defined in the fixed base frame while orientation is expressed in the end-effector frame. We denoted this configuration as ‘mixed frame robotic arms’. Consequently, conventional visual servo controllers which operate in a single frame of reference are incompatible with mixed frame ARAs. Therefore, we propose a mixed-frame visual servo control framework for ARAs. Moreover, we enlightened the task space kinematics of a mixed frame ARAs, which led us to the development of a novel “mixed frame Jacobian matrix”. The proposed framework was validated on a mixed frame JACO-2 7 DoF ARA using an adaptive proportional derivative controller for achieving image-based visual servoing (IBVS), which showed a significant increase of 31% in the convergence rate, outperforming conventional IBVS joint controllers, especially in the outstretched arm positions and near the base frame. Our Results determine the need for the mixed frame controller for deploying visual servo control on modern ARAs, that can inherently cater to the robotic arm’s joint limits, singularities, and self-collision problems.

Does One Size Fit All? A Case Study to Discuss Findings of an Augmented Hands-Free Robot Teleoperation Concept for People with and without Motor Disabilities

Hands-free robot teleoperation and augmented reality have the potential to create an inclusive environment for people with motor disabilities. It may allow them to teleoperate robotic arms to manipulate objects. However, the experiences evoked by the same teleoperation concept and augmented reality can vary significantly for people with motor disabilities compared to those without disabilities. In this paper, we report the experiences of Miss L., a person with multiple sclerosis, when teleoperating a robotic arm in a hands-free multimodal manner using a virtual menu and visual hints presented through the Microsoft HoloLens 2. We discuss our findings and compare her experiences to those of people without disabilities using the same teleoperation concept. Additionally, we present three learning points from comparing these experiences: a re-evaluation of the metrics used to measure performance, being aware of the bias, and considering variability in abilities, which evokes different experiences. We consider these learning points can be extrapolated to carrying human–robot interaction evaluations with mixed groups of participants with and without disabilities.

Abnormal actions detection of robotic arm via 3D convolution neural network and support vector data description

Robotic arms are powerful assistants in many industrial production environments, and they run periodically in accordance with preset actions to complete specified operations. However, they may act abnormally when encountering unexpected situation and then lead to unnecessary loss. Recognizing the abnormal actions of robotic arms through surveillance video can automatically help us to understand their operating status and discover possible abnormalities in time. We designed a deep learning architecture based on 3D convolution for abnormal action recognition. The 3D convolutional layer can extract the spatial and temporal features of the robotic arm movements from the video frame difference sequence. The features are compressed and streamlined by the maximum pooling layer to obtain concise and effective robotic arm action features. Finally, the fully connected layer is used to classify the features to recognize the abnormal robotic arm tasks. Support vector data description (SVDD) model is employed to detect abnormal actions of the robotic arm, and the well-trained SVDD model can distinguish the normal actions from the three kinds of abnormal actions with the Area Under Curve (AUC) 99.17% .

Intelligent Robotic Palletizer System

In the global wave of automation, logistics and manufacturing are indispensable and important industries. Among them, the related automatic warehousing system is even more urgently needed. There are quite a few cases of using robotic arms in the current industry cargo stacking operations. Traditional operations require engineers to plan the stacking path for the robotic arm. If the size of the object changes, it will take extra time to re-plan the work path. Therefore, in recent years, quite a lot of automatic palletizing software has been developed; however, none of it has a detection mechanism for stacking correctness and personnel safety. As a result, in this research, an intelligent robotic palletizer system is developed based on a self-developed symmetrical algorithm to stack the goods in a staggered arrangement to ensure the overall structure. Innovatively, it is proposed to check the arrangement status and warnings during the visual stack inspection to ensure the correctness of the stacking process. Besides, an AI algorithm is imported to ensure that personnel cannot enter the set dangerous area during the work of the robotic arm to improve safety during stacking. In addition to uploading the relevant data to the cloud database in real time, the stacking process combined database and vision system also provide users with real-time monitoring of system information.

DESIGN OF OPTIMAL INVERSE KINEMATIC SOLUTION FOR HUMANOID ROBOTIC ARMS

One of the main problems in robotics is the Inverse Kinematics (IK) problem. In this paper, three optimization algorithms are proposed to solve the IK of Humanoid Robotic Arms (HRAs). A Particle Swarm Optimization (PSO), Social Spider Optimization (SSO), and Black Hole Optimization (BHO) algorithms are proposed in order to optimize the parameters of the proposed IK. Also, in this paper, each optimization method is applied on both right and left arms to find the desired positions and required angles with a minimum error. Denavit-Hartenberg (D-H) method is used to design and simulate the mathematical model of HRAs for both arms in which each arm has five Degree Of Freedom (DOF). The HRAs model is tested for performance by several positions to be reached by both arms in the same time to find which optimization algorithm is better. Optimal solution obtained by SSO, PSO and BHO algorithms are evaluated and listed in comparison table between them. These optimization algorithms are assessed by calculating the Computational Time (CT) and Root Mean Squared Error (RMSE) for the absolute error vector of the positions. Calculation and simulation results showed that BHO algorithm is better than the other optimization algorithms from point of view of CT and RMSE. The worst RMSE is 0.0864 was calculated using PSO algorithm. But longer CT is 7.6521 second, which was calculated using SSO. While the best RMSE and shorter CT.are and 3.0156 second respectively were calculated by BHO algorithm. Moreover, in this paper, the Graphical User Interface (GUI) is designed and built for motional characteristics of the HRAs model in the Forward Kinematics (FK) and IK. The optimization algorithms are designed using MATLAB package facilities to simulate the HRAs model and the solution of IK problem.