scholarly journals Towards open-source LOD2 modelling using convolutional neural networks

2021 ◽  
Author(s):  
H. Muftah ◽  
T. S. L. Rowan ◽  
A. P. Butler
Keyword(s):  
Rainwater Harvesting ◽  
Three Dimensional ◽  
Semantic Segmentation ◽  
3D Models ◽  
Surface Model ◽  
Convolutional Network ◽  
Harris Corner ◽  
Roof Area ◽  
Minimum Height ◽  
Bounding Boxes

AbstractThe aim of this paper is to classify and segment roofs using vertical aerial imagery to generate three-dimensional (3D) models. Such models can be used, for example, to evaluate the rainfall runoff from properties for rainwater harvesting and in assessing solar energy and roof insulation options. Aerial orthophotos and building footprints are used to extract individual roofs and bounding boxes, which are then fed into one neural network for classification and then another for segmentation. The approach initially implements transfer learning on a pre-trained VGG16 model. The first step achieves an accuracy of 95.39% as well as a F1 score of 95%. The classified images are segmented using a fully convolutional network semantic segmentation model. The mask of the segmented roof planes is used to extract the coordinates of the roof edges and the nexus points using the Harris corner detector algorithm. The coordinates of the corners are then used to plot a 3D Level of Detail 2 (LOD2) representation of the building and the roof height is determined by calculating the maximum and minimum height of a Digital Surface Model LiDAR point cloud and known building height data. Subsequently the wireframe plot can be used to compute the roof area. This model achieved an accuracy of 80.2%, 96.1%, 96.0%, 85.1% and 91.1% for flat, hip, gable, cross-hip and mansard roofs, respectively.

scholarly journals Three-Dimensional Reconstruction of Structural Surface Model of Heritage Bridges Using UAV-Based Photogrammetric Point Clouds

2019 ◽  
Vol 11 (10) ◽  
pp. 1204 ◽  
Author(s):  
Yue Pan ◽  
Yiqing Dong ◽  
Dalei Wang ◽  
Airong Chen ◽  
Zhen Ye
Keyword(s):  
Classification Tree ◽  
Three Dimensional ◽  
Point Clouds ◽  
3D Models ◽  
Surface Model ◽  
Reconstruction Method ◽  
3D Point Clouds ◽  
Surface Models ◽  
Structural Surface ◽  
Better Than

Three-dimensional (3D) digital technology is essential to the maintenance and monitoring of cultural heritage sites. In the field of bridge engineering, 3D models generated from point clouds of existing bridges is drawing increasing attention. Currently, the widespread use of the unmanned aerial vehicle (UAV) provides a practical solution for generating 3D point clouds as well as models, which can drastically reduce the manual effort and cost involved. In this study, we present a semi-automated framework for generating structural surface models of heritage bridges. To be specific, we propose to tackle this challenge via a novel top-down method for segmenting main bridge components, combined with rule-based classification, to produce labeled 3D models from UAV photogrammetric point clouds. The point clouds of the heritage bridge are generated from the captured UAV images through the structure-from-motion workflow. A segmentation method is developed based on the supervoxel structure and global graph optimization, which can effectively separate bridge components based on geometric features. Then, recognition by the use of a classification tree and bridge geometry is utilized to recognize different structural elements from the obtained segments. Finally, surface modeling is conducted to generate surface models of the recognized elements. Experiments using two bridges in China demonstrate the potential of the presented structural model reconstruction method using UAV photogrammetry and point cloud processing in 3D digital documentation of heritage bridges. By using given markers, the reconstruction error of point clouds can be as small as 0.4%. Moreover, the precision and recall of segmentation results using testing date are better than 0.8, and a recognition accuracy better than 0.8 is achieved.

StyroStone: A protocol for scanning and extracting three-dimensional meshes of stone artefacts using Micro-CT scanners v1

2021 ◽  
Author(s):  
Dominik Göldner ◽  
Fotios Alexandros Karakostis ◽  
Armando Falcucci
Keyword(s):  
Three Dimensional ◽  
Computation Time ◽  
Ct Scanning ◽  
3D Models ◽  
Stone Tools ◽  
Surface Model ◽  
Micro Ct ◽  
Ct Scanner ◽  
Micro Computed Tomography ◽  
Ct Scanners

This protocol presents the first detailed step-by-step pipeline for the 3D scanning and post processing of large batches of lithic artefacts using a micro-computed tomography (micro-CT) scanner (i.e., a Phoenix v-tome-x S model by General Electronics MCC, Boston MA) and an Artec Space Spider scanner (Artec Inc., Luxembourg). This protocol was used to scan and analyze ca. 700 lithic artefacts from the Protoaurignacian layers at Fumane Cave in north-eastern Italy (Falcucci et al., in preparation). For this study several costly scanners and proprietary software packages were employed. Although it is not easy to find a low-budget alternative for the scanners, it is possible to use free and open-source software programs, such as 3D-Slicer (https://www.slicer.org/) or MorphoDig (https://morphomuseum.com/morphodig), to process CT data as well as MeshLab (Cignoni et al. 2008) to interact with the 3D models in general. However, if alternative software is used, the steps and their order described in this protocol might diverge significantly. A cost-effective alternative to create 3D models is digital photogrammetry using commercial cameras and freely available software like Meshroom (https://alicevision.org). Although photogrammetry is an affordable technique to create accurate 3D models of objects, this method might not be useful when scanning large batches of artefacts, as it will require a lot of computation time and processing capacity. Likewise, it could be difficult to generate accurate 3D models of very small and/or detailed tool shapes using 3D surface scanners because stone tools are often much smaller than the recommended minimum field of view. Similarly, the resolution of conventional medical CT scanners might not be sufficient to capture minor details of stone tools, such as the outline or dorsal scars. Thus, high-resolution micro-CT technology is the only reliable way to accurately capture the overall morphology of small stone tools. This protocol aims at providing the first detailed procedure dedicated to the scanning of small lithic implements for further three-dimensional analysis. Note that some of the steps must be repeated at different working stages throughout this protocol. In cases where a task must be done in the exact same way as described in a previous step, a reference to that step is provided. When slight changes were made, the step was modified and reported entirely. This protocol contains a few red and green colours (e.g., arrows or within-program colours) which might be perceived differently by people with dyschromatopsia. However, the display of these colours has been kept to a minimum. We recommend the reader to go over the entire protocol carefully, even if only some specific parts are required. A few points are in fact interdependent, and some of them must be applied simultaneously. Content: Part 1 – Styrofoam preparation Part 2 – Micro-CT scanning Part 3 – 3D model extraction of CT scanned stone artifacts using Avizo Part 4 – Cropping extracted surface model to separate Face A and B in Artec Studio Part 5 – Cropping Face A to separate the lines in Artec Studio Part 6 – Cropping each stone artefact from the lines in Artec Studio Part 7 – Virtually control measurements in MeshLab Part 8 – Artec scanning of larger artifacts Part 9 – Export meshes as non-binary ply models for successive analysis in geomorph Three-dimensional example (in ply format) of the effectivity of the StyroStone Protocol: You can download an example of one Styrofoam line in 3D obtained using our protocol to appreciate the result that can be achieved. We have selected a line where objects are characterized by different metric and morphological attributes. Notice the retouching well visible in the last five smaller artifacts (counting from the left when artifact are oriented with the dorsal face in front of the observer and the butt down), as well as the platforms and bulbs of all artifacts. For more information and examples, feel free to contact us!

scholarly journals Knowledge and Geo-Object Based Graph Convolutional Network for Remote Sensing Semantic Segmentation

Sensors ◽  
2021 ◽  
Vol 21 (11) ◽  
pp. 3848
Author(s):  
Wei Cui ◽  
Meng Yao ◽  
Yuanjie Hao ◽  
Ziwei Wang ◽  
Xin He ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Prior Knowledge ◽  
Contextual Information ◽  
Information Aggregation ◽  
Semantic Segmentation ◽  
Spatial Correlations ◽  
Convolutional Network ◽  
Object Based ◽  
Graph Neural Networks ◽  
Salt And Pepper

Pixel-based semantic segmentation models fail to effectively express geographic objects and their topological relationships. Therefore, in semantic segmentation of remote sensing images, these models fail to avoid salt-and-pepper effects and cannot achieve high accuracy either. To solve these problems, object-based models such as graph neural networks (GNNs) are considered. However, traditional GNNs directly use similarity or spatial correlations between nodes to aggregate nodes’ information, which rely too much on the contextual information of the sample. The contextual information of the sample is often distorted, which results in a reduction in the node classification accuracy. To solve this problem, a knowledge and geo-object-based graph convolutional network (KGGCN) is proposed. The KGGCN uses superpixel blocks as nodes of the graph network and combines prior knowledge with spatial correlations during information aggregation. By incorporating the prior knowledge obtained from all samples of the study area, the receptive field of the node is extended from its sample context to the study area. Thus, the distortion of the sample context is overcome effectively. Experiments demonstrate that our model is improved by 3.7% compared with the baseline model named Cluster GCN and 4.1% compared with U-Net.

scholarly journals RobotP: A Benchmark Dataset for 6D Object Pose Estimation

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1299
Author(s):  
Honglin Yuan ◽  
Tim Hoogenkamp ◽  
Remco C. Veltkamp
Keyword(s):  
Pose Estimation ◽  
Ground Truth ◽  
3D Models ◽  
Depth Image ◽  
Great Success ◽  
Estimation Algorithms ◽  
Depth Images ◽  
Object Pose Estimation ◽  
Image Pairs ◽  
Bounding Boxes

Deep learning has achieved great success on robotic vision tasks. However, when compared with other vision-based tasks, it is difficult to collect a representative and sufficiently large training set for six-dimensional (6D) object pose estimation, due to the inherent difficulty of data collection. In this paper, we propose the RobotP dataset consisting of commonly used objects for benchmarking in 6D object pose estimation. To create the dataset, we apply a 3D reconstruction pipeline to produce high-quality depth images, ground truth poses, and 3D models for well-selected objects. Subsequently, based on the generated data, we produce object segmentation masks and two-dimensional (2D) bounding boxes automatically. To further enrich the data, we synthesize a large number of photo-realistic color-and-depth image pairs with ground truth 6D poses. Our dataset is freely distributed to research groups by the Shape Retrieval Challenge benchmark on 6D pose estimation. Based on our benchmark, different learning-based approaches are trained and tested by the unified dataset. The evaluation results indicate that there is considerable room for improvement in 6D object pose estimation, particularly for objects with dark colors, and photo-realistic images are helpful in increasing the performance of pose estimation algorithms.

scholarly journals Dense cellular segmentation for EM using 2D–3D neural network ensembles

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Matthew D. Guay ◽  
Zeyad A. S. Emam ◽  
Adam B. Anderson ◽  
Maria A. Aronova ◽  
Irina D. Pokrovskaya ◽  
...  
Keyword(s):  
Blood Platelets ◽  
Semantic Segmentation ◽  
3D Models ◽  
Image Features ◽  
Cell Models ◽  
Whole Cells ◽  
Large Numbers ◽  
The Creation ◽  
Segmentation Task ◽  
Neural Network Ensembles

AbstractBiologists who use electron microscopy (EM) images to build nanoscale 3D models of whole cells and their organelles have historically been limited to small numbers of cells and cellular features due to constraints in imaging and analysis. This has been a major factor limiting insight into the complex variability of cellular environments. Modern EM can produce gigavoxel image volumes containing large numbers of cells, but accurate manual segmentation of image features is slow and limits the creation of cell models. Segmentation algorithms based on convolutional neural networks can process large volumes quickly, but achieving EM task accuracy goals often challenges current techniques. Here, we define dense cellular segmentation as a multiclass semantic segmentation task for modeling cells and large numbers of their organelles, and give an example in human blood platelets. We present an algorithm using novel hybrid 2D–3D segmentation networks to produce dense cellular segmentations with accuracy levels that outperform baseline methods and approach those of human annotators. To our knowledge, this work represents the first published approach to automating the creation of cell models with this level of structural detail.

scholarly journals Parallel Fully Convolutional Network for Semantic Segmentation

IEEE Access ◽  
2021 ◽  
Vol 9 ◽  
pp. 673-682
Author(s):  
Jian Ji ◽  
Xiaocong Lu ◽  
Mai Luo ◽  
Minghui Yin ◽  
Qiguang Miao ◽  
...  
Keyword(s):  
Semantic Segmentation ◽  
Convolutional Network ◽  
Fully Convolutional Network

scholarly journals Structured-light 3D scanning of exhibited historical clothing—a first-ever methodical trial and its results

2021 ◽  
Vol 9 (1) ◽  
Author(s):  
Jerzy Montusiewicz ◽  
Marek Miłosz ◽  
Jacek Kęsik ◽  
Kamil Żyła
Keyword(s):  
Cultural Heritage ◽  
Information Technologies ◽  
Structured Light ◽  
Three Dimensional ◽  
3D Models ◽  
3D Scanning ◽  
Dimensional Representation ◽  
3D Scanner ◽  
Three Dimensional Representation ◽  
3D Scanners

AbstractHistorical costumes are part of cultural heritage. Unlike architectural monuments, they are very fragile, which exacerbates the problems of their protection and popularisation. A big help in this can be the digitisation of their appearance, preferably using modern techniques of three-dimensional representation (3D). The article presents the results of the search for examples and methodologies of implementing 3D scanning of exhibited historical clothes as well as the attendant problems. From a review of scientific literature it turns out that so far practically no one in the world has made any methodical attempts at scanning historical clothes using structured-light 3D scanners (SLS) and developing an appropriate methodology. The vast majority of methods for creating 3D models of clothes used photogrammetry and 3D modelling software. Therefore, an innovative approach was proposed to the problem of creating 3D models of exhibited historical clothes through their digitalisation by means of a 3D scanner using structural light technology. A proposal for the methodology of this process and concrete examples of its implementation and results are presented. The problems related to the scanning of 3D historical clothes are also described, as well as a proposal how to solve them or minimise their impact. The implementation of the methodology is presented on the example of scanning elements of the Emir of Bukhara's costume (Uzbekistan) from the end of the nineteenth century, consisting of the gown, turban and shoes. Moreover, the way of using 3D models and information technologies to popularise cultural heritage in the space of digital resources is also discussed.

scholarly journals Comparative Analysis of Digital Models of Objects of Cultural Heritage Obtained by the “3D SLS” and “SfM” Methods

2021 ◽  
Vol 11 (12) ◽  
pp. 5321
Author(s):  
Marcin Barszcz ◽  
Jerzy Montusiewicz ◽  
Magdalena Paśnikowska-Łukaszuk ◽  
Anna Sałamacha
Keyword(s):  
Cultural Heritage ◽  
Low Cost ◽  
Three Dimensional ◽  
3D Models ◽  
Silk Road ◽  
Digital Models ◽  
Global Pandemic ◽  
University Of Technology ◽  
Three Dimensional Models ◽  
3D Digitisation

In the era of the global pandemic caused by the COVID-19 virus, 3D digitisation of selected museum artefacts is becoming more and more frequent practice, but the vast majority is performed by specialised teams. The paper presents the results of comparative studies of 3D digital models of the same museum artefacts from the Silk Road area generated by two completely different technologies: Structure from Motion (SfM)—a method belonging to the so-called low-cost technologies—and by Structured-light 3D Scanning (3D SLS). Moreover, procedural differences in data acquisition and their processing to generate three-dimensional models are presented. Models built using a point cloud were created from data collected in the Afrasiyab museum in Samarkand (Uzbekistan) during “The 1st Scientific Expedition of the Lublin University of Technology to Central Asia” in 2017. Photos for creating 3D models in SfM technology were taken during a virtual expedition carried out under the “3D Digital Silk Road” program in 2021. The obtained results show that the quality of the 3D models generated with SfM differs from the models from the technology (3D SLS), but they may be placed in the galleries of the vitrual museum. The obtained models from SfM do not have information about their size, which means that they are not fully suitable for archiving purposes of cultural heritage, unlike the models from SLS.

scholarly journals 3D-Volume Rendering of the Pelvis with Emphasis on Paraurethral Structures Based on MRI Scans and Comparisons between 3D Slicer and OsiriX®

2021 ◽  
Vol 45 (3) ◽  
Author(s):  
C. M. Durnea ◽  
S. Siddiqi ◽  
D. Nazarian ◽  
G. Munneke ◽  
P. M. Sedgwick ◽  
...  
Keyword(s):  
Image Processing ◽  
Secondary Analysis ◽  
Three Dimensional ◽  
3D Models ◽  
Clinical Role ◽  
Cross Sectional ◽  
3D Slicer ◽  
Mri Scans ◽  
3D Volume ◽  
Good Agreement

AbstractThe feasibility of rendering three dimensional (3D) pelvic models of vaginal, urethral and paraurethral lesions from 2D MRI has been demonstrated previously. To quantitatively compare 3D models using two different image processing applications: 3D Slicer and OsiriX. Secondary analysis and processing of five MRI scan based image sets from female patients aged 29–43 years old with vaginal or paraurethral lesions. Cross sectional image sets were used to create 3D models of the pelvic structures with 3D Slicer and OsiriX image processing applications. The linear dimensions of the models created using the two different methods were compared using Bland-Altman plots. The comparisons demonstrated good agreement between measurements from the two applications. The two data sets obtained from different image processing methods demonstrated good agreement. Both 3D Slicer and OsiriX can be used interchangeably and produce almost similar results. The clinical role of this investigation modality remains to be further evaluated.

scholarly journals Estimation of 6D Object Pose Using a 2D Bounding Box

Sensors ◽  
2021 ◽  
Vol 21 (9) ◽  
pp. 2939
Author(s):  
Yong Hong ◽  
Jin Liu ◽  
Zahid Jahangir ◽  
Sheng He ◽  
Qing Zhang
Keyword(s):  
Neural Network ◽  
Loss Function ◽  
Three Dimensional ◽  
Unit Vector ◽  
Prediction Algorithm ◽  
Computational Time ◽  
Bounding Box ◽  
Dimensional Unit ◽  
Bounding Boxes ◽  
Rgb Image

This paper provides an efficient way of addressing the problem of detecting or estimating the 6-Dimensional (6D) pose of objects from an RGB image. A quaternion is used to define an object′s three-dimensional pose, but the pose represented by q and the pose represented by -q are equivalent, and the L2 loss between them is very large. Therefore, we define a new quaternion pose loss function to solve this problem. Based on this, we designed a new convolutional neural network named Q-Net to estimate an object’s pose. Considering that the quaternion′s output is a unit vector, a normalization layer is added in Q-Net to hold the output of pose on a four-dimensional unit sphere. We propose a new algorithm, called the Bounding Box Equation, to obtain 3D translation quickly and effectively from 2D bounding boxes. The algorithm uses an entirely new way of assessing the 3D rotation (R) and 3D translation rotation (t) in only one RGB image. This method can upgrade any traditional 2D-box prediction algorithm to a 3D prediction model. We evaluated our model using the LineMod dataset, and experiments have shown that our methodology is more acceptable and efficient in terms of L2 loss and computational time.

Sign in / Sign up

Export Citation Format

Share Document