Towards Tangible Cultural Heritage Experiences—Enriching VR-based Object Inspection with Haptic Feedback

VR/AR technology is a key enabler for new ways of immersively experiencing cultural heritage artifacts based on their virtual counterparts obtained from a digitization process. In this article, we focus on enriching VR-based object inspection by additional haptic feedback, thereby creating tangible cultural heritage experiences. For this purpose, we present an approach for interactive and collaborative VR-based object inspection and annotation. Our system supports high-quality 3D models with accurate reflectance characteristics while additionally providing haptic feedback regarding shape features of the object based on a 3D printed replica. The digital object model in terms of a printable representation of the geometry as well as reflectance characteristics are stored in a compact and streamable representation on a central server, which streams the data to remotely connected users/clients. The latter can jointly perform an interactive inspection of the object in VR with additional haptic feedback through the 3D printed replica. Evaluations regarding system performance, visual quality of the considered models, as well as insights from a user study indicate an improved interaction, assessment, and experience of the considered objects.

OBJECT TRACKING CONTROL USING A GIMBAL MECHANISM

This work describes a control solution for real time object tracking in images acquired for a RPAS on an object inspection environment. This, controlling a 3-axis gimbal mechanism to control a camera orientation embedded to a RPAS, using its image processed for feedback. The objective of control is to maintain the target of interest at the center of the image plane. The proposed solution uses a YOLOv3 object detection model in order to detect the target object and determine, thru rotation matrices, the new desired angles to converge the object’s position to the center of the image. To compare results of the proposed control, a linear control was tuned using a linear PI algorithm. Simulation and practice experiments successfully tracked the desired object in real time using YOLOv3 in both control approaches presented.

ALGORITHMS OF SEAFLOOR INDUSTRIAL MACHINERY INSPECTION USING AUV

АНПА могут быть применены для автоматизированной инспекции объектов подводных добычных комплексов. При наличии априорной информации об объекте (его модели) целью обследования может быть детальная фотосъемка заданных фрагментов объекта. Для выхода АНПА к этим фрагментам используются алгоритмы точной навигационной привязки к объекту на базе анализа последовательности стереоизображений и имеющейся информации о расположении характерных точек объекта. В случае отсутствия априорной информации целью обследования может быть построение детальной 3D модели объекта (с помощью лазерного сканера или многолучевого эхолокатора). Для этого АНПА производит первоначальное обнаружение (локализацию) объекта. Далее его траектория формируется динамически от одной видовой позиции к другой по мере поступления новой информации об объекте. В качестве критерия при выборе очередной видовой позиции используется оценка ее информативности (т.е. объем получаемой новой информации об объекте). Итоговая модель объекта формируется по собранной информации с использованием методов фотограмметрии. В работе рассматриваются подходы и алгоритмы, которые могут быть использованы при обследовании объектов для этих двух случаев. Autonomous underwater vehicles (AUV) can be used for automatized inspection of the objects of marine mining complexes. If a priori information about the object (or its model) is available, the examination can be aimed at detailed photographing the given fragments of the object. Routing AUV to these fragments relies on algorithms of precise navigation referencing to the object based on analysis of the sequence of stereo images and available information about specific points of the object. In case of the necessary information absence, the inspection may target the goal of building the detailed 3D model of the object (using a laser scanner or multibeam echosounder). In that case, AUV performs the initial detection (localization) of the object at first. Then its motion trajectory is generated dynamically from one view position to another as new information about the object becomes available. A criterion of the viewing position selection used is its informational value (i.e., the amount of further information received about the object). The final model is built using gathered information processed with methods of photogrammetry. The paper considers approaches and algorithms applicable for object inspection in two above described cases.