Controlling mechanical arms using 3D coordinates gotten from the AR technology

How to control the movement of the mechanical arm is a challenging topic recently. It requires much time and energy to program every joint. This paper proposes a new method based on 3D coordinates of the human arm joints, which helps the mechanical arm to imitate the movement of human arms and only needs to calculate angles of human arms’ joints according to the 3D coordinates. Here in this process the key is to get the information of 3D coordinates of human arms. This paper applies an AR (Augmented Reality) application of body detection with LiDAR on iPad Pro 2020 to get 3D coordinates. Then the calculated angle information is transferred from the host computer to the Arduino board controlling the movement of the mechanical arm. Eventually the same movements will be made by the mechanical arm and the human arm.