scholarly journals An overview of ongoing point cloud compression standardization activities: video-based (V-PCC) and geometry-based (G-PCC)

2020 ◽  
Vol 9 ◽  
Author(s):  
D. Graziosi ◽  
O. Nakagami ◽  
S. Kuma ◽  
A. Zaghetto ◽  
T. Suzuki ◽  
...  
Keyword(s):  
Augmented Reality ◽  
Cultural Heritage ◽  
Motion Picture ◽  
Point Cloud ◽  
Autonomous Driving ◽  
Data Representation ◽  
Media Content ◽  
International Standard ◽  
3D Data ◽  
Immersive Media

Abstract This article presents an overview of the recent standardization activities for point cloud compression (PCC). A point cloud is a 3D data representation used in diverse applications associated with immersive media including virtual/augmented reality, immersive telepresence, autonomous driving and cultural heritage archival. The international standard body for media compression, also known as the Motion Picture Experts Group (MPEG), is planning to release in 2020 two PCC standard specifications: video-based PCC (V-CC) and geometry-based PCC (G-PCC). V-PCC and G-PCC will be part of the ISO/IEC 23090 series on the coded representation of immersive media content. In this paper, we provide a detailed description of both codec algorithms and their coding performances. Moreover, we will also discuss certain unique aspects of point cloud compression.

scholarly journals ARTS, an AR Tourism System, for the Integration of 3D Scanning and Smartphone AR in Cultural Heritage Tourism and Pedagogy

Sensors ◽  
2019 ◽  
Vol 19 (17) ◽  
pp. 3725 ◽  
Author(s):  
Naai-Jung Shih ◽  
Pei-Huang Diao ◽  
Yi Chen
Keyword(s):  
Augmented Reality ◽  
Cultural Heritage ◽  
Point Cloud ◽  
Heritage Tourism ◽  
Urban Fabric ◽  
Heritage Site ◽  
System Usability ◽  
3D Scans ◽  
Graphic Module ◽  
Cultural Heritage Tourism

Interactions between cultural heritage, tourism, and pedagogy deserve investigation in an as-built environment under a macro- or micro-perspective of urban fabric. The heritage site of Shih Yih Hall, Lukang, was explored. An Augmented Reality Tourism System (ARTS) was developed on a smartphone-based platform for a novel application scenario using 3D scans converted from a point cloud to a portable interaction size. ARTS comprises a real-time environment viewing module, a space-switching module, and an Augmented Reality (AR) guide graphic module. The system facilitates scenario initiations, projection and superimposition, annotation, and interface customization, with software tools developed using ARKit® on the iPhone XS Max®. The three-way interaction between urban fabric, cultural heritage tourism, and pedagogy was made possible through background block-outs and an additive or selective display. The illustration of the full-scale experience of the smartphone app was made feasible for co-relating the cultural dependence of urban fabric on tourism. The great fidelity of 3D scans and AR scenes act as a pedagogical aid for students or tourists. A Post-Study System Usability Questionnaire (PSSUQ) evaluation verified the usefulness of ARTS.

scholarly journals Generating Point Cloud from Measurements and Shapes Based on Convolutional Neural Network: An Application for Building 3D Human Model

2019 ◽  
Vol 2019 ◽  
pp. 1-15 ◽  
Author(s):  
Mau Tung Nguyen ◽  
Thanh Vu Dang ◽  
Minh Kieu Tran Thi ◽  
Pham The Bao
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Point Cloud ◽  
Data Representation ◽  
Human Model ◽  
Average Error ◽  
Shape Information ◽  
Hierarchical Learning ◽  
3D Data ◽  
3D Shapes

It has been widely known that 3D shape models are comprehensively parameterized using point cloud and meshes. The point cloud particularly is much simpler to handle compared with meshes, and it also contains the shape information of a 3D model. In this paper, we would like to introduce our new method to generating the 3D point cloud from a set of crucial measurements and shapes of importance positions. In order to find the correspondence between shapes and measurements, we introduced a method of representing 3D data called slice structure. A Neural Networks-based hierarchical learning model is presented to be compatible with the data representation. Primary slices are generated by matching the measurements set before the whole point cloud tuned by Convolutional Neural Network. We conducted the experiment on a 3D human dataset which contains 1706 examples. Our results demonstrate the effectiveness of the proposed framework with the average error 7.72% and fine visualization. This study indicates that paying more attention to local features is worthwhile when dealing with 3D shapes.

scholarly journals CATCHA: Real-Time Camera Tracking Method for Augmented Reality Applications in Cultural Heritage Interiors

2018 ◽  
Vol 7 (12) ◽  
pp. 479 ◽  
Author(s):  
Piotr Siekański ◽  
Jakub Michoński ◽  
Eryk Bunsch ◽  
Robert Sitnik
Keyword(s):  
Augmented Reality ◽  
Cultural Heritage ◽  
Real Time ◽  
Point Cloud ◽  
Frame Rate ◽  
Actual State ◽  
Camera Tracking ◽  
Time Model ◽  
Pose Tracking ◽  
Camera Pose

Camera pose tracking is a fundamental task in Augmented Reality (AR) applications. In this paper, we present CATCHA, a method to achieve camera pose tracking in cultural heritage interiors with rigorous conservatory policies. Our solution is real-time model-based camera tracking according to textured point cloud, regardless of its registration technique. We achieve this solution using orthographic model rendering that allows us to achieve real-time performance, regardless of point cloud density. Our developed algorithm is used to create a novel tool to help both cultural heritage restorers and individual visitors visually compare the actual state of a culture heritage location with its previously scanned state from the same point of view in real time. The provided application can directly achieve a frame rate of over 15 Hz on VGA frames on a mobile device and over 40 Hz using remote processing. The performance of our approach is evaluated using a model of the King’s Chinese Cabinet (Museum of King Jan III’s Palace at Wilanów, Warsaw, Poland) that was scanned in 2009 using the structured light technique and renovated and scanned again in 2015. Additional tests are performed on a model of the Al Fresco Cabinet in the same museum, scanned using a time-of-flight laser scanner.

scholarly journals Symmetry Analysis of Oriental Polygonal Pagodas Using 3D Point Clouds for Cultural Heritage

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1228
Author(s):  
Ting On Chan ◽  
Linyuan Xia ◽  
Yimin Chen ◽  
Wei Lang ◽  
Tingting Chen ◽  
...  
Keyword(s):  
Cultural Heritage ◽  
Unmanned Aerial Vehicle ◽  
Point Cloud ◽  
Geometric Model ◽  
Digital Camera ◽  
Point Clouds ◽  
Symmetry Analysis ◽  
3D Point Clouds ◽  
Transmission Towers ◽  
Aerial Vehicle

Ancient pagodas are usually parts of hot tourist spots in many oriental countries due to their unique historical backgrounds. They are usually polygonal structures comprised by multiple floors, which are separated by eaves. In this paper, we propose a new method to investigate both the rotational and reflectional symmetry of such polygonal pagodas through developing novel geometric models to fit to the 3D point clouds obtained from photogrammetric reconstruction. The geometric model consists of multiple polygonal pyramid/prism models but has a common central axis. The method was verified by four datasets collected by an unmanned aerial vehicle (UAV) and a hand-held digital camera. The results indicate that the models fit accurately to the pagodas’ point clouds. The symmetry was realized by rotating and reflecting the pagodas’ point clouds after a complete leveling of the point cloud was achieved using the estimated central axes. The results show that there are RMSEs of 5.04 cm and 5.20 cm deviated from the perfect (theoretical) rotational and reflectional symmetries, respectively. This concludes that the examined pagodas are highly symmetric, both rotationally and reflectionally. The concept presented in the paper not only work for polygonal pagodas, but it can also be readily transformed and implemented for other applications for other pagoda-like objects such as transmission towers.

Volumetric Video in Augmented Reality Applications for Museological Narratives

2021 ◽  
Vol 14 (2) ◽  
pp. 1-20
Author(s):  
Néill O’dwyer ◽  
Emin Zerman ◽  
Gareth W. Young ◽  
Aljosa Smolic ◽  
Siobhán Dunne ◽  
...  
Keyword(s):  
Augmented Reality ◽  
Cultural Heritage ◽  
Applied Research ◽  
Ease Of Use ◽  
Museum Visitors ◽  
The Public ◽  
Research Article ◽  
Nature Of Information ◽  
Live Action ◽  
Capture Technique

Cross-reality technologies are quickly establishing themselves as commonplace platforms for presenting objects of historical, scientific, artistic, and cultural interest to the public. In this space, augmented reality (AR) is notably successful in delivering cultural heritage applications, including architectural and environmental heritage reconstruction, exhibition data management and representation, storytelling, and exhibition curation. Generally, it has been observed that the nature of information delivery in applications created for narrating exhibitions tends to be informative and formal. Here we report on the assessment of a pilot scene for a prototype AR application that attempts to break this mold by employing a humorous and playful mode of communication. This bespoke AR experience harnessed the cutting-edge live-action capture technique of volumetric video to create a digital tour guide that playfully embellished the museological experience of the museum visitors. This applied research article consists of measuring, presenting, and discussing the appeal, interest, and ease of use of this ludic AR storytelling strategy mediated via AR technology in a cultural heritage context.

scholarly journals Revealing Hidden Local Cultural Heritage through a Serious Escape Game in Outdoor Settings

Information ◽  
2020 ◽  
Vol 12 (1) ◽  
pp. 10
Author(s):  
Stavroula Tzima ◽  
Georgios Styliaras ◽  
Athanasios Bassounas
Keyword(s):  
Virtual Reality ◽  
Augmented Reality ◽  
Cultural Heritage ◽  
Physical Environment ◽  
Local History ◽  
Educational Context ◽  
Virtual Reality Technology ◽  
The Real ◽  
Hybrid Type ◽  
Game Mechanics

Escape Rooms are presently considered a very popular social entertainment activity, with increasing popularity in education field, since they are considered capable of stimulating the interest of players/students and enhancing learning. The combined game mechanics have led to blended forms of Escape Rooms, the Serious Escape Games (SEGs) and the hybrid type of Escape Rooms that uses Augmented Reality (AR)/Virtual Reality technology, a type that is expected to be widely used in the future. In the current study, the MillSecret is presented, a multi-player Serious Escape Game about local cultural heritage, where the players must solve a riddle about the cultural asset of watermills. MillSecret uses AR technology and it was designed to be conducted in the real-physical environment and in an informal educational context. The paper describes the game, its implementation, the playing process, and its evaluation, which aimed to study the feasibility of game conduction in outdoor settings and the views and experience of players with the game, the local cultural heritage and local history. Evaluation results reveal, among other findings, a very positive first feedback from players that allows us to further evolve the development of the game.

scholarly journals Parallel Point Clouds: Hybrid Point Cloud Generation and 3D Model Enhancement via Virtual–Real Integration

2021 ◽  
Vol 13 (15) ◽  
pp. 2868
Author(s):  
Yonglin Tian ◽  
Xiao Wang ◽  
Yu Shen ◽  
Zhongzheng Guo ◽  
Zilei Wang ◽  
...  
Keyword(s):  
Point Cloud ◽  
Closed Loop ◽  
Real Data ◽  
Point Clouds ◽  
Autonomous Driving ◽  
Training Model ◽  
Labor Cost ◽  
Open Loop ◽  
Training Dataset ◽  
Data Annotation

Three-dimensional information perception from point clouds is of vital importance for improving the ability of machines to understand the world, especially for autonomous driving and unmanned aerial vehicles. Data annotation for point clouds is one of the most challenging and costly tasks. In this paper, we propose a closed-loop and virtual–real interactive point cloud generation and model-upgrading framework called Parallel Point Clouds (PPCs). To our best knowledge, this is the first time that the training model has been changed from an open-loop to a closed-loop mechanism. The feedback from the evaluation results is used to update the training dataset, benefiting from the flexibility of artificial scenes. Under the framework, a point-based LiDAR simulation model is proposed, which greatly simplifies the scanning operation. Besides, a group-based placing method is put forward to integrate hybrid point clouds, via locating candidate positions for virtual objects in real scenes. Taking advantage of the CAD models and mobile LiDAR devices, two hybrid point cloud datasets, i.e., ShapeKITTI and MobilePointClouds, are built for 3D detection tasks. With almost zero labor cost on data annotation for newly added objects, the models (PointPillars) trained with ShapeKITTI and MobilePointClouds achieved 78.6% and 60.0% of the average precision of the model trained with real data on 3D detection, respectively.

Sign in / Sign up

Export Citation Format

Share Document