Physics-Based Graphics Models in 3D Synthetic Environments as Autonomous Vision-Based Inspection Testbeds

Manual visual inspection of civil infrastructure is high-risk, subjective, and time-consuming. The success of deep learning and the proliferation of low-cost consumer robots has spurred rapid growth in research and application of autonomous inspections. The major components of autonomous inspection include data acquisition, data processing, and decision making, which are usually studied independently. However, for robust real-world applicability, these three aspects of the overall process need to be addressed concurrently with end-to-end testing, incorporating scenarios such as variations in structure type, color, damage level, camera distance, view angle, lighting, etc. Developing real-world datasets that span all these scenarios is nearly impossible. In this paper, we propose a framework to create a virtual visual inspection testbed using 3D synthetic environments that can enable end-to-end testing of autonomous inspection strategies. To populate the 3D synthetic environment with virtual damaged buildings, we propose the use of a non-linear finite element model to inform the realistic and automated visual rendering of different damage types, the damage state, and the material textures of what are termed herein physics-based graphics models (PBGMs). To demonstrate the benefits of the autonomous inspection testbed, three experiments are conducted with models of earthquake damaged reinforced concrete buildings. First, we implement the proposed framework to generate a new large-scale annotated benchmark dataset for post-earthquake inspections of buildings termed QuakeCity. Second, we demonstrate the improved performance of deep learning models trained using the QuakeCity dataset for inference on real data. Finally, a comparison of deep learning-based damage state estimation for different data acquisition strategies is carried out. The results demonstrate the use of PBGMs as an effective testbed for the development and validation of strategies for autonomous vision-based inspections of civil infrastructure.

Transmission line surface defect detection method based on uav autonomous inspection

The existing transmission line surface defect detection methods have the problem of incomplete image data set, resulting in a low recognition success rate. A transmission line surface defect detection method based on uav autonomous inspection is designed. The safety of power grid operation is evaluated, the local linearization process is transformed into linear equation expression, the image data set is obtained by uav autonomous inspection, the transmission line state is judged, the corresponding constraint conditions are set, the type of transmission line surface defects are identified, the number of image poles and towers is matched, and the detection mode is optimized by edge detection algorithm. Experimental results: The average recognition success rate of the transmission line surface defect detection method in this paper and the other two detection methods is 59.89%, 51.89% and 52.03%, proving that the transmission line surface defect detection method integrating UAV technology inspection has a wider application space.

Physics-Based Graphics Models in 3D Synthetic Environments Enabling Autonomous Vision-Based Structural Inspections

Manual visual inspections typically conducted after an earthquake are high-risk, subjective, and time-consuming. Delays from inspections often exacerbate the social and economic impact of the disaster on affected communities. Rapid and autonomous inspection using images acquired from unmanned aerial vehicles offer the potential to reduce such delays. Indeed, a vast amount of re-search has been conducted toward developing automated vision-based methods to assess the health of infrastructure at the component and structure level. Most proposed methods typically rely on images of the damaged structure, but seldom consider how the images were acquired. To achieve autonomous inspections, methods must be evaluated in a comprehensive end-to-end manner, incorporating both data acquisition and data processing. In this paper, we leverage recent advances in computer generated imagery (CGI) to construct a 3D synthetic environment for simulation of post-earthquake inspections that allows for comprehensive evaluation and valida-tion of autonomous inspection strategies. A critical issue is how to simulate and subsequently render the damage in the structure after an earthquake. To this end, a high-fidelity nonlinear finite element model is incorporated in the synthetic environment to provide a representation of earthquake-induced damage; this finite element model, combined with photo-realistic rendering of the damage, is termed herein a physics-based graphics models (PBGM). The 3D synthetic en-vironment with PBGMs provide a comprehensive end-to-end approach for development and validation of autonomous post-earthquake strategies using UAVs, including: (i) simulation of path planning of virtual UAVs and image capture under different environmental conditions; (ii) au-tomatic labeling of captured images, potentially providing an infinite amount of data for training deep neural networks; (iii) availability of the ground truth damage state from the results of the finite-element simulation; and (iv) direct comparison of different approaches to autonomous as-sessments. Moreover, the synthetic data generated has the potential to be used to augment field datasets. To demonstrate the efficacy of PBGMs, models of reinforced concrete moment-frame buildings with masonry infill walls are examined. The 3D synthetic environment employing PBGMs is shown to provide an effective testbed for development and validation of autonomous vision-based post-earthquake inspections that can serve as an important building block for ad-vancing autonomous data to decision frameworks.

Tracking Control of Unmanned Aerial Vehicle for Power Line Inspection

This work presents some results about power transmission line tracking control and a full autonomous inspection using a quadrotor helicopter. The presented in this paper power line autonomous inspection allows detecting power line defects caused by thunderstorms, corrosion, insulator malfunctions, and same time monitoring of vegetation under the power line corridor. Traditional inspection is performed by helicopters equipped with high-resolution cameras or by direct visual examination carried out by highly skilled staff climbing over de-energized power lines. However, the visual inspection is time-expensive and costly. Moreover, due to regulatory constraints, the helicopters cannot cover narrow mountainous areas. Unmanned aerial vehicles (UAV) are an attractive alternative for power line inspection. In this work, a mathematical model for the quadrotor helicopter used in the autonomous inspection is presented. The model is successfully evaluated through simulations and flight experiments. Next, the construction of a quadrotor helicopter system and its application to power line autonomous inspection is introduced. Simulation and experimental results demonstrate the efficiency and applicability of that system. The results of this research are in the process of implementation for regular inspection of electrical transmission lines.