Intelligent Digital Twins and Augmented Reality in Inspection and Maintenance

Objective/Scope Simplify and semi-automate the creation of Intelligent Digital Twins available on an augmented hybrid solution (on/offline) based on the latest mobile solutions and wearable technologies (Smart Glasses) communicating over Bluetooth. The main objective was to implement an interactive environment of digital twins through Augmented Reality (AR) to improve decision-making, enhance performance and optimize efficiencies in Inspection and Maintenance. Methods, Procedures, Process A breakthrough algorithm enabled to semi-automate the creation of 3D models by providing a platform of Intelligent Digital Twins to connect to various disparate systems and use Artificial Intelligence-driven augmentation techniques to contextualize and enrich the model with Tag numbers from the asset register. An application has been developed for Smart Glasses by transforming the Digital Twins into sectioned files to enable a performant AR experience and an on/offline syncing functionality with the mobile solution over Bluetooth. Furthermore, the 3D model was embedded within the mobile application for visual support during Inspection and Maintenance. Results, Observations, Conclusions As a result, the following products have been developed to improve efficiency and productivity by leveraging innovative technologies: Application for Intelligent Digital Twins: Single Source of Truth for managing and maintaining Intelligent Digital Twins using breakthrough technology to semi-automate the creation of 3D models from Point Cloud data. A Mark-Up Tool and a Contextualization Editor were developed on the promise of Digital Twins to introduce a communication, collaboration, and planning tool. Furthermore, the 3D model has been embedded within the mobile application environment for visual support on the field tablets. Application for Smart Glasses (AR Headset): An application has been developed for the latest AR Headset to enable an Augmented Reality experience and a Mark-Up tool with Bluetooth communication to the mobile application as well as to support a hybrid on/offline syncing functionality as well as data capturing on the fly. With its basic functionality of hands-free video streaming and communication functionality, the AR Headset enabled a Remote Pre-Commissioning, Certification, and Audit with vendors and independent certifiers during the pandemic. Application Programming Interfaces (API): API interfaces were established to various source systems like the Computerized Maintenance Management System, Inspection Data Management System, Drawing System, and the Process Information System to integrate and visualize the big data within the platform of Intelligent Digital Twins. Novel/Additive Information The use of automation and artificial intelligence (AI) driven technologies for creating Intelligent Digital Twins is a key enabler for global scalability and accelerates creating the company's digital backbone during the digital transformation. Enhancing this technology with an Augmented Reality experience, including mark-up annotations, provides a solid basis for data insights, data-driven decision making, and performance optimization.

The Effect of Insight Questions Inventory and Visual Support Strategies on Carer-Reported Quality of Life for Children With Cerebral Palsy and Perceptual Visual Dysfunction in Nigeria: A Randomized Controlled Trial

Structured clinical history question inventories have previously been used to try and elicit symptoms of perceptual visual dysfunction (PVD) in children with cerebral palsy (CP) in different settings. Earlier studies have suggested that PVD may affect quality of life and specific habilitational strategies, linked to inventory responses, may improve quality of life. Through an RCT, based on a community based sample of children with CP in Cross River State, Nigeria, we aimed to determine if a structured history inventory such as the Insight question inventory (IQI) and associated tailored visual support strategies (IQI VSS) for the management of those children who have PVD, can improve quality of life and is superior to standard therapy. Children with CP were recruited by the key informant method and confirmed by clinical examination. The parent reported IQI was used to identify children with PVD. Primary outcome measures were both Pediatric Quality of Life 4.0 Generic (PedsQL 4.0 Generic) and Pediatric Quality of Life 3.0 Cerebral Palsy (PedsQL 3.0 CP) scale scores. Children were enrolled with a parallel arm allocation to either IQI and IQI VSS or to standard therapy for CP. Children were followed up for 6 weeks with weekly phone call session and the questionnaires repeated at the end of the 6 weeks’ period. Results show that the children in the treatment group (n = 191) showed no significantly different change between baseline and follow up in quality of life (PedsQL 4.0 Generic p = 0.943: and PedsQL-CP 3.0 p = 0.287), compared to the control group. There was suggestion of a better improvement (p = 0.035) in the PedsQL 3.0 CP subscale of speech and communication for the intervention group. The use of IQI VSS for the treatment of PVD in children with CP in this population does not show any superiority over current standard CP management in terms of overall quality of life. However, there was some evidence of improvement in quality of life in the area of speech and communication. Further research and refinement of these management method is required.Clinical Trial Registration:www.ClinicalTrials.gov, identifier [PACTR20161200188] 6396.

The influence of visual supports and motivation on motor performance of the MABC-2 for Chinese school-aged children with autism spectrum disorder

The influences of including visual supports and strategies to increase motivation for children with autism spectrum disorder (ASD) in motor assessments were examined. 97 children with ASD and 117 age-matched typically developing (TD) children performed the Movement Assessment Battery for Children, Second Edition (MABC-2) under traditional, visual support, motivation, and visual support plus motivation protocols. Results showed that children with ASD elicited lower MABC-2 scores than TD children. Moreover, in children with ASD, the visual support protocol, but not the motivation protocol, produced higher scores on ball and balance skills than the traditional protocol. These findings indicated that children with ASD were developmentally delayed in motor skills; however, their performance may be improved by including visual supports in motor assessments.

HearIt: Auditory-Cue-Based Audio Playback Control to Facilitate Information Browsing in Lecture Audio

Students often utilize audio media during online or offline courses. However, lecture audio data are mostly unstructured and extensive, so they are more challenging in information browsing (i.e., chaining, linking, extraction, and evaluation of relevant information). Conventional time-level skip control is limited in auditory information browsing because it is hard to identify the current position and context. This paper presents HearIt, which provides semantic-level skip control with auditory cues for auditory information browsing. With HearIt, users can efficiently change the playback position in the paragraph-level. Furthermore, two auditory cues (positional cue and topical cue) help grasp the current playback and its context without additional visual support. We conducted a pilot study with the prototype of HearIt, and the results show its feasibility and design implications for future research.

Semi-Automatic Visual Support System with Drone for Teleoperated Construction Robot

In this paper, we describe a semi-automatic viewpoint moving system that employs a drone to provide visual assistance to the operator of a teleoperated robot. The objective of this system is to improve the operational efficiency and reduce the mental load of the operator. The operator changes the position of the drone through an interface to acquire the optimal assist image for teleoperation. We confirmed through an evaluation experiment that, in comparison with our previous study in which the final positions of the drone were determined in advance, the proposed method improves the operational accuracy and reduces the mental load of the operator.