Fast, Accurate, and Automated 3D Reconstruction Using a Depth Camera Mounted on an Industrial Robot

2021 ◽  
Author(s):  
Rishi Malhan ◽  
Rex Jomy Joseph ◽  
Prahar M. Bhatt ◽  
Brual Shah ◽  
Satyandra K. Gupta
Keyword(s):  
3D Reconstruction ◽  
Cycle Time ◽  
Industrial Robot ◽  
Low Cost ◽  
Point Clouds ◽  
Industrial Robots ◽  
Depth Camera ◽  
Reconstruction Process ◽  
Depth Cameras ◽  
Manufacturing Applications

Abstract 3D reconstruction technology is used in a wide variety of applications. Currently, automatically creating accurate pointclouds for large parts requires expensive hardware. We are interested in using low-cost depth cameras mounted on commonly available industrial robots to create accurate pointclouds for large parts automatically. Manufacturing applications require fast cycle times. Therefore, we are interested in speeding up the 3D reconstruction process. We present algorithmic advances in 3D reconstruction that achieve a sub-millimeter accuracy using a low-cost depth camera. Our system can be used to determine a pointcloud model of large and complex parts. Advances in camera calibration, cycle time reduction for pointcloud capturing, and uncertainty estimation are made in this work. We continuously capture point-clouds at an optimal camera location with respect to part distance during robot motion execution. The redundancy in pointclouds achieved by the moving camera significantly reduces errors in measurements without increasing cycle time. Our system produces sub-millimeter accuracy.

ALGORITHMS FOR IMPROVING SPEED AND ACCURACY OF AUTOMATED 3D RECONSTRUCTION WITH A DEPTH CAMERA MOUNTED ON AN INDUSTRIAL ROBOT

2021 ◽  
pp. 1-13
Author(s):  
Rishi K. Malhan ◽  
Rex Jomy Joseph ◽  
Prahar Bhatt ◽  
Brual Shah ◽  
Satyandra K. Gupta
Keyword(s):  
3D Reconstruction ◽  
Cycle Time ◽  
Industrial Robot ◽  
Low Cost ◽  
Industrial Robots ◽  
Depth Camera ◽  
Reconstruction Process ◽  
Depth Cameras ◽  
Camera Location ◽  
Manufacturing Applications

Abstract 3D reconstruction technology is used in a wide variety of applications. Currently, automatically creating accurate pointclouds for large parts requires expensive hardware. We are interested in using low-cost depth cameras mounted on commonly available industrial robots to create accurate pointclouds for large parts automatically. Manufacturing applications require fast cycle times. Therefore, we are interested in speeding up the 3D reconstruction process. We present algorithmic advances in 3D reconstruction that achieve a sub-millimeter accuracy using a low-cost depth camera. Our system can be used to determine a pointcloud model of large and complex parts. Advances in camera calibration, cycle time reduction for pointcloud capturing, and uncertainty estimation are made in this work. We continuously capture pointclouds at an optimal camera location with respect to part distance during robot motion execution. The redundancy in pointclouds achieved by the moving camera significantly reduces errors in measurements without increasing cycle time. Our system produces sub-millimeter accuracy.

Online 3D Reconstruction in Underwater Environment Using a Low-Cost Depth Camera

2021 ◽  
pp. 237-244
Author(s):  
Luigi Scarfone ◽  
Rosario Aiello ◽  
Umberto Severino ◽  
Loris Barbieri ◽  
Fabio Bruno
Keyword(s):  
3D Reconstruction ◽  
Low Cost ◽  
Depth Camera ◽  
Underwater Environment

scholarly journals Vision System of Mobile Robot Combining Binocular and Depth Cameras

2017 ◽  
Vol 2017 ◽  
pp. 1-11 ◽  
Author(s):  
Yuxiang Yang ◽  
Xiang Meng ◽  
Mingyu Gao
Keyword(s):  
Mobile Robot ◽  
3D Reconstruction ◽  
Trajectory Planning ◽  
Stereo Matching ◽  
Vision System ◽  
Depth Camera ◽  
Structural Constraints ◽  
Depth Cameras ◽  
Stereo Cameras ◽  
Binocular Stereo

In order to optimize the three-dimensional (3D) reconstruction and obtain more precise actual distances of the object, a 3D reconstruction system combining binocular and depth cameras is proposed in this paper. The whole system consists of two identical color cameras, a TOF depth camera, an image processing host, a mobile robot control host, and a mobile robot. Because of structural constraints, the resolution of TOF depth camera is very low, which difficultly meets the requirement of trajectory planning. The resolution of binocular stereo cameras can be very high, but the effect of stereo matching is not ideal for low-texture scenes. Hence binocular stereo cameras also difficultly meet the requirements of high accuracy. In this paper, the proposed system integrates depth camera and stereo matching to improve the precision of the 3D reconstruction. Moreover, a double threads processing method is applied to improve the efficiency of the system. The experimental results show that the system can effectively improve the accuracy of 3D reconstruction, identify the distance from the camera accurately, and achieve the strategy of trajectory planning.

Towards Remote Teleoperation of a Semi-Autonomous Mobile Manipulator System in Machine Tending Tasks

2019 ◽  
Author(s):  
Vivek Annem ◽  
Pradeep Rajendran ◽  
Shantanu Thakar ◽  
Satyandra K. Gupta
Keyword(s):  
Material Handling ◽  
Point Clouds ◽  
Mobile Manipulator ◽  
Depth Cameras ◽  
Level Of Automation ◽  
Volume Production ◽  
Manufacturing Applications ◽  
High Level ◽  
The Given ◽  
Mobile Base

Abstract Increasing the level of automation in material handling tasks in small volume production operations can improve human productivity and overall manufacturing system performance. In this paper, we present a teleoperated mobile manipulator system that can be used for tending machines and transporting parts in manufacturing applications. The remotely located human operator can interact with the semi-autonomous mobile manipulator by giving it high level instructions. We have incorporated several sensors on the system to ensure safe teleoperation where the operator gives only high level motion goals to the mobile manipulator, such as waypoints for mobile base motion, and interactive marker poses for the manipulator motion. The point clouds from multiple depth cameras are used for mapping the environment. The robot plans for autonomous motions between the given way-points ensuring that the resulting motions are collision-free. We have conducted case studies with two different types of parts to be extracted from a 3D printer. The system is tested by multiple users. They were successful in completing tasks in a reasonable amount of time using our interface.

scholarly journals The feasibility of using a low-cost depth camera for 3D scanning in mass customization

2019 ◽  
Vol 9 (1) ◽  
pp. 450-458
Author(s):  
Juho-Pekka Virtanen ◽  
Kim-Niklas Antin ◽  
Matti Kurkela ◽  
Hannu Hyyppä
Keyword(s):  
Additive Manufacturing ◽  
Mass Customization ◽  
Computer Aided Design ◽  
Low Cost ◽  
Measurement Techniques ◽  
Depth Camera ◽  
Camera System ◽  
Depth Cameras ◽  
Sharp Edges ◽  
Aided Design

AbstractBy combining additive manufacturing with 3D measurement techniques, tailored production work-flows that include the digitizing of existing components, computer-aided design, and tool-free manufacturing of the customized parts can be envisioned, potentially reducing the costs of mass customization. The introduction of affordable depth cameras has greatly increased the consumer availability of 3D measuring. We present the application of an affordable depth camera for the 3D digitizing of existing components, the utilization of the produced data in the design process, and finally, the production of the designed component with additive manufacturing. The capabilities of the affordable depth camera system are evaluated by comparing it with photogrammetric 3D reconstruction, revealing issues in smaller geometric details and sharp edges.

HIGH-SPEED INDUSTRIAL REAL-TIME NETWORK OF CYBERPHYSICAL SYSTEMS

2019 ◽  
pp. 46-52
Author(s):  
A. A. Zelensky
Keyword(s):  
Real Time ◽  
Cycle Time ◽  
High Speed ◽  
High Efficiency ◽  
Industrial Robot ◽  
Field Tests ◽  
Industrial Robots ◽  
Third Party ◽  
Industrial Equipment ◽  
Industrial Automation

The construction of a high-speed industrial real-time network based on FPGA (Field-Programmable Gate Array) for the control of machines and industrial robots is considered. A brief comparative analysis of the performance of the implemented Ethernet-based Protocol with industrial protocols of other leading manufacturers is made. The aim of the research and development of its own industrial automation Protocol was to reduce the dependence on third-party real-time protocols based on Ethernet for controlling robots, machines and technological equipment. In the course of the study, the requirements for the network of the motion control system of industrial equipment were analyzed. In order to synchronize different network nodes and provide short exchange cycle time, an industrial managed switch was developed, as well as a specialized hardware controller for processing Ethernet packets for end devices, presented as a IP-core. A key feature of the developed industrial network is that the data transmission in it is completely determined, and the exchange cycle time for each of the network devices can be configured individually. High efficiency and performance of implemented network devices became possible due to the use of hardware solutions based on FPGAs. All solutions described in the article as part of a modular digital system have been successfully tested in the control of machines and industrial robot. The results of field tests show that the use of FPGAs and soft processors with specialized peripheral IP-blocks can significantly reduce the tact of managing industrial equipment through the use of hardware computing structures, which indicates the promise of the proposed approach for solving industrial automation tasks.

3D reconstruction of human head based on consumer RGB-D sensors

2020 ◽  
pp. 2050061
Author(s):  
Zihan Liu ◽  
Guanghong Gong ◽  
Ni Li ◽  
Zihao Yu
Keyword(s):  
3D Reconstruction ◽  
Low Cost ◽  
Three Dimensional ◽  
Human Head ◽  
Point Clouds ◽  
Microsoft Kinect ◽  
Head Model ◽  
Registration Method ◽  
Three Dimensional Modeling ◽  
Texture Generation

Three-dimensional (3D) reconstruction of a human head with high precision has promising applications in scientific research, product design and other fields. However, it still faces resistance from two factors. One is inaccurate registration caused by symmetrical distribution of head feature points, and the other is economic burden due to high-accuracy sensors. Research on 3D reconstruction with portable consumer RGB-D sensors such as the Microsoft Kinect has been highlighted in recent years. Based on our multi-Kinect system, a precise and low-cost three-dimensional modeling method and its system implementation are introduced in this paper. A registration method for multi-source point clouds is provided, which can reduce the fusion differences and reconstruct the head model accurately. In addition, a template-based texture generation algorithm is presented to generate a fine texture. The comparison and analysis of our experiments show that our method can reconstruct a head model in an acceptable time with less memory and better effect.

scholarly journals IMPACT OF LEVEL OF DETAILS IN THE 3D RECONSTRUCTION OF TREES FOR MICROCLIMATE MODELING

2016 ◽  
Vol XLI-B8 ◽  
pp. 257-264
Author(s):  
E. Bournez ◽  
T. Landes ◽  
M. Saudreau ◽  
P. Kastendeuch ◽  
G. Najjar
Keyword(s):  
3D Reconstruction ◽  
Urban Areas ◽  
Urban Climate ◽  
Point Clouds ◽  
Climatic Conditions ◽  
Urban Heat Islands ◽  
Natural Phenomenon ◽  
Intensity Increase ◽  
Reconstruction Process ◽  
Level Of Details

In the 21st century, urban areas undergo specific climatic conditions like urban heat islands which frequency and intensity increase over the years. Towards the understanding and the monitoring of these conditions, vegetation effects on urban climate are studied. It appears that a natural phenomenon, the evapotranspiration of trees, generates a cooling effect in urban environment. In this work, a 3D microclimate model is used to quantify the evapotranspiration of trees in relation with their architecture, their physiology and the climate. These three characteristics are determined with field measurements and data processing. Based on point clouds acquired with terrestrial laser scanner (TLS), the 3D reconstruction of the tree wood architecture is performed. Then the 3D reconstruction of leaves is carried out from the 3D skeleton of vegetative shoots and allometric statistics. With the aim of extending the simulation on several trees simultaneously, it is necessary to apply the 3D reconstruction process on each tree individually. However, as well for the acquisition as for the processing, the 3D reconstruction approach is time consuming. Mobile laser scanners could provide point clouds in a faster way than static TLS, but this implies a lower point density. Also the processing time could be shortened, but under the assumption that a coarser 3D model is sufficient for the simulation. In this context, the criterion of level of details and accuracy of the tree 3D reconstructed model must be studied. In this paper first tests to assess their impact on the determination of the evapotranspiration are presented.

scholarly journals A Methodology for Industrial Robot Calibration Based on Measurement Sub-Regions

2021 ◽  
Author(s):  
Juan Sebastian Toquica ◽  
José Maurı́cio Motta
Keyword(s):  
Industrial Robot ◽  
Low Cost ◽  
Kinematic Model ◽  
Measurement Data ◽  
Industrial Applications ◽  
Industrial Robots ◽  
Robot Calibration ◽  
High Precision Measurement ◽  
Measurement Systems ◽  
Robot Accuracy

Abstract This paper proposes a methodology for calibration of industrial robots that uses a concept of measurement sub-regions, allowing low-cost solutions and easy implementation to meet the robot accuracy requirements in industrial applications. The solutions to increasing the accuracy of robots today have high-cost implementation, making calibration throughout the workplace in industry a difficult and unlikely task. Thus, reducing the time spent and the measured workspace volume of the robot end-effector are the main benefits of the implementation of the sub-region concept, ensuring sufficient flexibility in the measurement step of robot calibration procedures. The main contribution of this article is the proposal and discussion of a methodology to calibrate robots using several small measurement sub-regions and gathering the measurement data in a way equivalent to the measurements made in large volume regions, making feasible the use of high-precision measurement systems but limited to small volumes, such as vision-based measurement systems. The robot calibration procedures were simulated according to the literature, such that results from simulation are free from errors due to experimental setups as to isolate the benefits of the measurement proposal methodology. In addition, a method to validate the analytical off-line kinematic model of industrial robots is proposed using the nominal model of the robot supplier incorporated into its controller.

Design of an Inertial Measuring Unit for Control of Robotic Devices

2019 ◽  
Vol 952 ◽  
pp. 313-322 ◽  
Author(s):  
Emil Škultéty ◽  
Elena Pivarčiová ◽  
Ladislav Karrach
Keyword(s):  
Technology Development ◽  
Industrial Robot ◽  
Low Cost ◽  
Cost Savings ◽  
Industrial Robots ◽  
Measurement Unit ◽  
Mems Sensor ◽  
Wide Range ◽  
Robotic Devices ◽  
Robotic Application

Industrial robots are increasingly used to automate technological processes, such as machining, welding, paint coating, assembly, etc. Automation rationalizes material flows, integrates production facilities and reduces the need for manufacturing inventory, provides cost savings for human maintenance. Technology development and growing competition have an influence on production growth and increase of product quality, and thus the new possibilities in innovation of industrial robot are searched for. One of the possibilities is applying of an inertial navigation system into robot control. This article focuses on new trends in manufacturing technology: design of Inertial Measurement Unit (IMU) for a robotic application control. The Arduino platform is used for the IMU as a hardware solution. The advantage of this platform is low cost and wide range of sensors and devices that are compatible with this platform. For scanning, the MEMS sensor MPU6050 is used, which includes a 3-axis gyroscope and an accelerometer in one chip. New trends in manufacturing facilities, especially robotics innovation and automation, will enable the productivity to grow in production processes.

Sign in / Sign up

Export Citation Format

Share Document