Model-Based Multi-Objective Reinforcement Learning by a Reward Occurrence Probability Vector

This chapter describes solving multi-objective reinforcement learning (MORL) problems where there are multiple conflicting objectives with unknown weights. Previous model-free MORL methods take large number of calculations to collect a Pareto optimal set for each V/Q-value vector. In contrast, model-based MORL can reduce such a calculation cost than model-free MORLs. However, previous model-based MORL method is for only deterministic environments. To solve them, this chapter proposes a novel model-based MORL method by a reward occurrence probability (ROP) vector with unknown weights. The experimental results are reported under the stochastic learning environments with up to 10 states, 3 actions, and 3 reward rules. The experimental results show that the proposed method collects all Pareto optimal policies, and it took about 214 seconds (10 states, 3 actions, 3 rewards) for total learning time. In future research directions, the ways to speed up methods and how to use non-optimal policies are discussed.

Modeling, Model Reduction, and Control of a Hands-Free Two-Wheeled Self-Balancing Scooter

Designing a two-wheeled self-balancing scooter involves in the synergistic approach of multidisciplinary engineering fields with mutual relationships of power transmission, mass transmission, and information transmission. The scooter consists of several subsystems and forms a large-scale system. The mathematical models are in the complex algebraic and differential equations in the form of high dimension. The complexity of its controller renders difficulties in its realization due to the limit of iteration period of real time control. Routh model reduction technique is employed to convert the original high-dimensional mathematical model into a simplified lower dimensional form. The modeling is derived using a unified variational method for both mechanical and electrical subsystems of the scooter, and for the electronic components equivalent circuit method is adopted. Simulations of the system response are based on the reduced model and its control design. A prototype is developed and realized with Matlab-Labview simulation and control environment.

Designing a Robot for Manufacturing Fiberglass Reinforced Plastic (FRP) Molded Grating

The use of robotics in the industrial environment has, in general, very similar goals. Because of productivity requirements, or due to reliability, industries have been constantly equipping their floor with robots. In that sense, the chapter observed—in a fiberglass company—the chance of using a robot to execute a boring and repetitive task. The task mentioned is, actually, the manufacturing of fiberglass reinforced plastic (FRP) molded grating. To confirm the possibility of using a robot to this job, a cost and time analysis was made about the whole molded gratings manufacturing process. Afterward, research about robotics was taken in parallel with the conception of the robot (named “roving-robot”). Calculations were made to the mechanical project of the robot. Applying computer-aided design (CAD), technical drawing and bill of materials were generated to permit the robot assembling. All of these project steps are presented in this chapter.

An Efficient Learning of Neural Networks to Acquire Inverse Kinematics Model

In this chapter, effective learning approach of inverse kinematics using neural networks with efficient weights update ability has been presented for a serial link structure and industrial robot. Generally, in making neural networks learn a relation among multi inputs and outputs, a desired training data set prepared in advance is used. The training data set consists of multiple pairs of input and output vectors. The input layer receives each input vector for forward computation, and it is compared with the yielded vector from the output layer. The time required for the learning process of the neural networks depends on the number of total weights in the neural networks and that of the input-output pairs in the training data set.

Cable-Driven Robots in Physical Rehabilitation

This chapter deals with cable-driven robots when applied in physical rehabilitation. In general, neurorehabilitation is limited to physical therapy that is delivered by clinicians and potentially augmented by robotic tools to facilitate neurorehabilitation and to reduce the consequences of central nervous system injury. Among the robotic tools for rehabilitation can be considered the cable-driven manipulators. First, this chapter presents the upper and lower human limbs movements. The main rehabilitation robots are presented as exoskeletons and cable-driven manipulators. After, the cable-driven manipulators theory is introduced focusing on considerations for robot design in rehabilitation and control with safe human-machine interaction. Experimental examples with different cable-driven robot's structures are presented so that this chapter suggests that these structures can be used as a complement to conventional therapies and not as a substitute. Finally, this chapter presents the clinical evidence in cable-driven robots when applied in physical rehabilitation.

Autonomous Surgical Robotics at Task and Subtask Levels

The revolution of minimally invasive procedures had a significant influence on surgical practice, opening the way to laparoscopic surgery, then evolving into robotics surgery. Teleoperated master-slave robots, such as the da Vinci Surgical System, has become a standard of care during the last few decades, performing over a million procedures per year worldwide. Many believe that the next big step in the evolution of surgery is partial automation, which would ease the cognitive load on the surgeon, making them possible to pay more attention on the critical parts of the intervention. Partial and sequential introduction and increase of autonomous capabilities could provide a safe way towards Surgery 4.0. Unfortunately, autonomy in the given environment, consisting mostly of soft organs, suffers from grave difficulties. In this chapter, the current research directions of subtask automation in surgery are to be presented, introducing the recent advances in motion planning, perception, and human-machine interaction, along with the limitations of the task-level autonomy.

Hybrid Dynamic Modelling and Bioinspired Control Based on Central Pattern Generator of Biped Robotic Gait

This chapter presents the theoretical foundations and methodology to develop a bioinspired hybrid control architecture for a biped robotic device that reproduces gait and human motor control strategies with the ability to adapt the trajectory to environmental conditions. The objective is to design robotic devices (such as exoskeletons), through the functional integration of hybrid dynamic system modeling (event-driven and continuous dynamics) with efficient and robust conventional control techniques and bioinspired control algorithms, with a near-natural human gait pattern. The human gait cycle is modeled as a hybrid dynamic using a finite state machine (FSM). The gait trajectories are to be generated in such a way that they will be capable of adapting to disturbances in the path followed by the robotic device; this will be achieved using a neuronal oscillator that simulates the behavior of a central pattern generator (CPG).

Framework for Quick and Intuitive Programming of Robot Applications

This chapter proposes a framework to make the programming of cobots faster, user-friendly and flexible for assembly tasks. The work focusses on an industrial case of a small (10kg) air compressor and investigates the technologies that can be used to automate this task with human-robot collaboration. To this end, the framework takes a radically different approach at the motion stack level and integrates the cobot with a constraint-based robot programming paradigm that enhances the robot programming possibilities. Additionally, the framework takes inputs from the operator via speech recognition and computer vision to increase the intuitiveness of the programing process. An implementation is made with focus on industrial robustness and the results show that this framework is a promising approach for the overall goal of achieving flexible assembly in the factories by making robot programming faster and intuitive.

Intelligent Processes in Automated Production Involving Industry 4.0 Technologies and Artificial Intelligence

The chapter presents different case studies involving technology upgrading involving Industry 4.0 technologies and artificial intelligence. The work analyzes four cases of study of industry projects related to manufacturing process of kitchen, tank production, pasta production, and electronic welding check. All the cases of study concern the analysis of engineered processes and the inline implementation of image vision techniques. The chapter discusses other topics involved in the production process such as augmented reality, quality prediction and predictive maintenance. The classic methodologies to map production processes are matched with innovative technologies of image segmentation and data mining predicting defects, machine failures, and product quality. The goal of the chapter is to prove how the combination of image processing techniques, data mining approaches, process simulation, chart process modeling, and process reengineering can constitute a scientific research project in industry research.

Industrial Exoskeletons With Gravity Compensation Elements

The chapter approaches the issues of modeling the process of load lifting by a person while wearing an exoskeleton. The classification of existing gravitational compensation systems for industrial exoskeletons is shown, as well as examples of its use. A mathematical model of lifting a person's load in the exoskeleton is presented, as well as numerical parameters are calculated. It is shown that the introduction of an elastic element reduces the level of energy consumption during work, and can also facilitate the level of the worker. Industrial exoskeleton prototype design is presented. A particular focus is given to studying the influence of the gravity compensator on the magnitude of the moments generated by the electric drives of the hip and knee joints. It is shown that the use of gravity compensators enables to reduce significantly the load on electric drives.