INTEGRATION OF BIM AND MOBILE AUGMENTED REALITY IN THE AECO DOMAIN

BIM (Building Information Modeling) is increasingly present in a wide range of applications (architecture, engineering, construction, land use planning, utility management, etc.). BIM allows better management of projects through precise planning, communication and collaboration between several stakeholders as well as facilitating the monitoring of project operations. The emergence of Augmented Reality (AR) technology allows the superposition of (2D, 3D) information directly on the physical world, so generating immersive, interactive and enriching experiences for the user. To take advantages of BIM and AR potential in the interaction and the intuitive management in AECO (Architecture, Engineering, Construction and Operation) projects, we propose a BIM-based AR workflow through an application called "EasyBIM". This latter allows access and interaction with a BIM model through functionalities for measurement, data consultation, collaboration, visualization and integration of information from sensors. The application is developed for mobile platforms (tablet, smartphone), and has as input an IFC file (Industry Foundation Classes). Promising test results show that the developed solution can be easily integrated into a BIM context for several use cases: marketing, collaboration, site monitoring, facility management, etc.

The Feasibility of Using UAV Structure from Motion Photogrammetry to Extract HBIM of the Great Ziggurat of UR

Culture heritage reflects nation’s legacy and therefore should be protected from damage in order to pass it to future generations. Recently, such protection can be applied by 3D digitization techniques such as conservation, restoration, documentation, etc. The 3D digitalization of heritage assets has encountered numerous focus in the last two decades due to the development in data capturing techniques and technological advancement in 3D remote sensing (RS) approaches such as photogrammetry and laser scanning. However, the abundance of 3D information resources and spatial data modelling and analysis methods have urged stakeholders to adopt intelligent 3D data management system so called Building Information Modelling (BIM) to facilitate data approaching and management. Historic Building Information Model (HBIM) is a special case of the BIM system, however it reflects the possibility to apply the BIM technology to the historical and heritage buildings. In this research, Structure from motion (SfM) photogrammetric routine based on Unmanned Aerial Vehicles (UAVs) images was applied to build HBIM of the Great Ziggurat of Ur in the south of Iraq. Based on the 3D geometric and texturing information extracted through photogrammetry and the historical information provided, virtual reconstruction has been carried out using (HBIM) technology. This was achieved by applying realistic materials and texturing information in order to document the building, which is directed to investigate the feasibility of implementing image-based 3D modelling within HBIM environments. Restoring the missing parts of the Ziggurat temple was also a focus of this research by implementing reverse engineering methodologies based on available information considered within the extracted HBIM. This can successfully represent a complete virtual reality model and a management information system of the Ziggurat building to be passed to future generations. The work also includes data assessment and validation with the as-built model generated from reference measurements within Computer Aided Design (CAD) environment.

Machine Learning-Based Lie Detector Applied to a Novel Annotated Game Dataset

Lie detection is considered a concern for everyone in their day-to-day life, given its impact on human interactions. Thus, people normally pay attention to both what their interlocutors are saying and to their visual appearance, including the face, to find any signs that indicate whether or not the person is telling the truth. While automatic lie detection may help us to understand these lying characteristics, current systems are still fairly limited, partly due to lack of adequate datasets to evaluate their performance in realistic scenarios. In this work, we collect an annotated dataset of facial images, comprising both 2D and 3D information of several participants during a card game that encourages players to lie. Using our collected dataset, we evaluate several types of machine learning-based lie detectors in terms of their generalization, in person-specific and cross-application experiments. We first extract both handcrafted and deep learning-based features as relevant visual inputs, then pass them into multiple types of classifier to predict respective lie/non-lie labels. Subsequently, we use several metrics to judge the models’ accuracy based on the models predictions and ground truth. In our experiment, we show that models based on deep learning achieve the highest accuracy, reaching up to 57% for the generalization task and 63% when applied to detect the lie to a single participant. We further highlight the limitation of the deep learning-based lie detector when dealing with cross-application lie detection tasks. Finally, this analysis along the proposed datasets would potentially be useful not only from the perspective of computational systems perspective (e.g., improving current automatic lie prediction accuracy), but also for other relevant application fields, such as health practitioners in general medical counselings, education in academic settings or finance in the banking sector, where close inspections and understandings of the actual intentions of individuals can be very important.

Calibration model and numerical simulation of a 7-hole probe based on pressure–velocity parameterization equation

As a sensor of a flow field detection system, a 7-hole probe can detect the flow field velocity and retrieve three-dimensional (3D) information of the flow field. Owing to its simple structure and strong environmental adaptability, it is particularly important to calibrate it when it is widely used in turbine machinery, aerospace, and other fields. To detect the 3D flow field in the middle atmosphere, a novel calibration method based on the potential flow theory is designed using a hemispherical 7-hole probe. The hemispherical 7-hole probe was numerically calibrated through numerical simulation, and the coefficients of the calibration equation are provided. In comparison with the traditional 7-hole probe calibration method, the calibration process is significantly shortened while maintaining good measurement accuracy. The velocity error was less than 5% and the angle error was approximately 0.5°.

Retrieval of 3D information in X-ray dark-field imaging with a large field of view

X-ray dark-field imaging is a widely researched imaging technique, with many studies on samples of very different dimensions and at very different resolutions. However, retrieval of three-dimensional (3D) information for human thorax sized objects has not yet been demonstrated. We present a method, similar to classic tomography and tomosynthesis, to obtain 3D information in X-ray dark-field imaging. Here, the sample is moved through the divergent beam of a Talbot–Lau interferometer. Projections of features at different distances from the source seemingly move with different velocities over the detector, due to the cone beam geometry. The reconstruction of different focal planes exploits this effect. We imaged a chest phantom and were able to locate different features in the sample (e.g. the ribs, and two sample vials filled with water and air and placed in the phantom) to corresponding focal planes. Furthermore, we found that image quality and detectability of features is sufficient for image reconstruction with a dose of 68 μSv at an effective pixel size of $$0.357 \times {0.357}\,\mathrm{mm}^{2}$$ 0.357 × 0.357 mm 2 . Therefore, we successfully demonstrated that the presented method is able to retrieve 3D information in X-ray dark-field imaging.

Scaling the U-net: segmentation of biodegradable bone implants in high-resolution synchrotron radiation microtomograms

Highly accurate segmentation of large 3D volumes is a demanding task. Challenging applications like the segmentation of synchrotron radiation microtomograms (SRμCT) at high-resolution, which suffer from low contrast, high spatial variability and measurement artifacts, readily exceed the capacities of conventional segmentation methods, including the manual segmentation by human experts. The quantitative characterization of the osseointegration and spatio-temporal biodegradation process of bone implants requires reliable, and very precise segmentation. We investigated the scaling of 2D U-net for high resolution grayscale volumes by three crucial model hyper-parameters (i.e., the model width, depth, and input size). To leverage the 3D information of high-resolution SRμCT, common three axes prediction fusing is extended, investigating the effect of adding more than three axes prediction. In a systematic evaluation we compare the performance of scaling the U-net by intersection over union (IoU) and quantitative measurements of osseointegration and degradation parameters. Overall, we observe that a compound scaling of the U-net and multi-axes prediction fusing with soft voting yields the highest IoU for the class “degradation layer”. Finally, the quantitative analysis showed that the parameters calculated with model segmentation deviated less from the high quality results than those obtained by a semi-automatic segmentation method.

Three-dimensional Tomography of Coseismic Ionospheric Disturbances from the 2016 West Sumatera Earthquake

Global Navigation Satellite System (GNSS) is a navigation system that uses satellite signals to determine its position, which consists of several satellites arranged in a constellation system. GNSS transmits signals to receivers on Earth. The GNSS receiver determines the user’s position, speed, and time by processing the signals transmitted by the satellites. The initial purpose of launching the GNSS was for navigation purposes, but along with its development, GNSS can be used for the purposes of observing deformation of the earth’s crust and in studying the atmosphere. The delayed wave data when passing through the ionosphere can be used to obtain Total Electron Content (TEC) values which then used to study ionospheric disturbances. Ionospheric disturbances are caused by various phenomena, the most common one is the ionospheric disturbances caused by the induction of acoustic and gravitational waves excited by co seismic crustal motions from large earthquakes. Ionospheric disturbances that happened before an earthquake are called Pre-seismic Ionospheric Disturbances and those that occur after an earthquake are called Co-seismic Ionospheric Disturbances (CID). Most studies of ionospheric disturbances still provide information on the timing and value of TEC anomalies in 2D form. Therefore, in this study, a 3D ionosphere profile modelling using computed 3D tomography will be carried out. The 3D information provided is in the form of time, ionosphere altitude and TEC anomaly value by utilizing GNSS data. The TEC anomaly value is obtained from the calculation of linear combination of the ionosphere. This study aims to obtain a spatial and temporal analysis of the CID caused by the West Sumatra Earthquake on March 2, 2016.

Machine Learning-Based Plant Detection Algorithms to Automate Counting Tasks Using 3D Canopy Scans

This study tested whether machine learning (ML) methods can effectively separate individual plants from complex 3D canopy laser scans as a prerequisite to analyzing particular plant features. For this, we scanned mung bean and chickpea crops with PlantEye (R) laser scanners. Firstly, we segmented the crop canopies from the background in 3D space using the Region Growing Segmentation algorithm. Then, Convolutional Neural Network (CNN) based ML algorithms were fine-tuned for plant counting. Application of the CNN-based (Convolutional Neural Network) processing architecture was possible only after we reduced the dimensionality of the data to 2D. This allowed for the identification of individual plants and their counting with an accuracy of 93.18% and 92.87% for mung bean and chickpea plants, respectively. These steps were connected to the phenotyping pipeline, which can now replace manual counting operations that are inefficient, costly, and error-prone. The use of CNN in this study was innovatively solved with dimensionality reduction, addition of height information as color, and consequent application of a 2D CNN-based approach. We found there to be a wide gap in the use of ML on 3D information. This gap will have to be addressed, especially for more complex plant feature extractions, which we intend to implement through further research.

Generating stable molecules using imitation and reinforcement learning

Chemical space is routinely explored by machine learning methods to discover interesting molecules, before time-consuming experimental synthesizing is attempted. However, these methods often rely on a graph representation, ignoring 3D information necessary for determining the stability of the molecules. We propose a reinforcement learning approach for generating molecules in cartesian coordinates allowing for quantum chemical prediction of the stability. To improve sample-efficiency we learn basic chemical rules from imitation learning on the GDB-11 database to create an initial model applicable for all stoichiometries. We then deploy multiple copies of the model conditioned on a specific stoichiometry in a reinforcement learning setting. The models correctly identify low energy molecules in the database and produce novel isomers not found in the training set. Finally, we apply the model to larger molecules to show how reinforcement learning further refines the imitation learning model in domains far from the training data.