Occlusion Handling in Augmented Reality Using Background-Foreground Segmentation and Projective Geometry

2005 ◽  
Vol 14 (3) ◽  
pp. 264-277 ◽  
Author(s):  
Hee Lin Wang ◽  
Kuntal Sengupta ◽  
Pankaj Kumar ◽  
Rajeev Sharma
Keyword(s):  
Augmented Reality ◽  
Real Time ◽  
Structural Model ◽  
Real Life ◽  
3D Models ◽  
Video Stream ◽  
Foreground Segmentation ◽  
Camera Motion ◽  
3D Objects ◽  
Active Research

Developing a seamless merging of real and virtual image streams and 3D models is an active research topic in augmented reality (AR). We propose a method for real-time augmentation of real videos with 2D and 3D objects by addressing the occlusion issue in an unique fashion. For virtual planar objects (such as images), the 2D overlay is automatically overlaid in a planar region selected by the user in the video. The overlay is robust to arbitrary camera motion. Furthermore, a unique background-foreground segmentation algorithm renders this augmented overlay as part of the background if it coincides with foreground objects in the video stream, giving the impression that it is occluded by foreground objects. The proposed technique does not require multiple cameras, camera calibration, use of fiducials, or a structural model of the scene to work. Extending the work further, we propose a novel method of augmentation by using trifocal tensors to augment 3D objects in 3D scenes to similar effect and implement it in real time as a proof of concept. We show several results of the successful working of our algorithm in real-life situations. The technique works on a real-time video from a USB camera, Creative Webcam III, onaPIV1.6GHz system without any special hardware support.

Geologic Modelling Using Augmented Reality

2021 ◽  
Author(s):  
Phathompat Boonyasaknanon ◽  
Raymond Pols ◽  
Katja Schulze ◽  
Robert Rundle
Keyword(s):  
Augmented Reality ◽  
Real Time ◽  
3D Models ◽  
New Approach ◽  
Domain Experts ◽  
3D Objects ◽  
Track Record ◽  
Initial Work ◽  
Geologic Model ◽  
Time Critical

Abstract An augmented reality (AR) system is presented which enhances the real-time collaboration of domain experts involved in the geologic modeling of complex reservoirs. An evaluation of traditional techniques is compared with this new approach. The objective of geologic modeling is to describe the subsurface as accurately and in as much detail as possible given the available data. This is necessarily an iterative process since as new wells are drilled more data becomes available which either validates current assumptions or forces a re-evaluation of the model. As the speed of reservoir development increases there is a need for expeditious updates of the subsurface model as working with an outdated model can lead to costly mistakes. Common practice is for a geologist to maintain the geologic model while working closely with other domain experts who are frequently not co-located with the geologist. Time-critical analysis can be hampered by the fact that reservoirs, which are inherently 3D objects, are traditionally viewed with 2D screens. The system presented here allows the geologic model to be rendered as a hologram in multiple locations to allow domain experts to collaborate and analyze the reservoir in real-time. Collaboration on 3D models has not changed significantly in a generation. For co-located personnel the approach is to gather around a 2D screen. For remote personnel the approach has been sharing a model through a 2D screen along with video chat. These approaches are not optimal for many reasons. Over the years various attempts have been tried to enhance the collaboration experience and have all fallen short. In particular virtual reality (VR) has been seen as a solution to this problem. However, we have found that augmented reality (AR) is a much better solution for many subtle reasons which are explored in the paper. AR has already acquired an impressive track record in various industries. AR will have applications in nearly all industries. For various historical reasons, the uptake for AR is much faster in some industries than others. It is too early to tell whether the use of augmented reality in geological applications will be transformative, however the results of this initial work are promising.

scholarly journals Real Time Shadow Mapping for Augmented Reality Photorealistic Rendering

2019 ◽  
Vol 9 (11) ◽  
pp. 2225 ◽  
Author(s):  
Francesco Osti ◽  
Gian Maria Santi ◽  
Gianni Caligiana
Keyword(s):  
Augmented Reality ◽  
Real Time ◽  
Real Life ◽  
Shadow Mapping ◽  
Photorealistic Rendering ◽  
Real Time Image ◽  
Image Capturing ◽  
Microsoft Hololens ◽  
The Right ◽  
Time Image

In this paper, we present a solution for the photorealistic ambient light render of holograms into dynamic real scenes, in augmented reality applications. Based on Microsoft HoloLens, we achieved this result with an Image Base Lighting (IBL) approach. The real-time image capturing that has been designed is able to automatically locate and position directional lights providing the right illumination to the holograms. We also implemented a negative “shadow drawing” shader that contributes to the final photorealistic and immersive effect of holograms in real life. The main focus of this research was to achieve a superior photorealism through the combination of real-time lights placement and negative “shadow drawing” shader. The solution was evaluated in various Augmented Reality case studies, from classical ones (using Vuforia Toolkit) to innovative applications (using HoloLens).

Placing Three-Dimensional Models in an Uncalibrated Single Image of an Architectural Scene

2004 ◽  
Vol 13 (6) ◽  
pp. 692-707 ◽  
Author(s):  
Sara Keren ◽  
Ilan Shimshoni ◽  
Ayellet Tal
Keyword(s):  
Augmented Reality ◽  
Three Dimensional ◽  
3D Models ◽  
Automatic Extraction ◽  
Single Image ◽  
Vanishing Points ◽  
Correct Position ◽  
3D Objects ◽  
Position And Orientation ◽  
Three Dimensional Models

This paper discusses the problem of inserting 3D models into a single image. The main focus of the paper is on the accurate recovery of the camera's parameters, so that 3D models can be inserted in the “correct” position and orientation. The paper addresses two issues. The first is an automatic extraction of the principal vanishing points from an image. The second is a theoretical and an experimental analysis of the errors. To test the concept, a system that “plants” virtual 3D objects in the image was implemented. It was tested on many indoor augmented-reality scenes. Our analysis and experiments have shown that errors in the placement of the objects are unnoticeable.

AN S-PI VISION-BASED TRACKING SYSTEM FOR OBJECT MANIPULATION IN AUGMENTED REALITY

2015 ◽  
Vol 75 (4) ◽  
Author(s):  
Ajune Wanis Ismail ◽  
Mark Bilinghust ◽  
Mohd Shahrizal Sunar
Keyword(s):  
Augmented Reality ◽  
Real Time ◽  
Tracking System ◽  
Object Manipulation ◽  
3D Graphics ◽  
Virtual Objects ◽  
3D Objects ◽  
Time Calibration ◽  
Detection Stage ◽  
Vision Algorithm

In this paper, we describe a new tracking approach for object handling in Augmented Reality (AR). Our approach improves the standard vision-based tracking system during marker extraction and its detection stage. It transforms a unique tracking pattern into set of vertices which are able to perform interaction such as translate, rotate, and copy. This is based on arobust real-time computer vision algorithm that tracks a paddle that a person uses for input. A paddle pose pattern is constructed in a one-time calibration process and through vertex-based calculation of the camera pose relative to the paddle we can show 3D graphics on top of it. This allows the user to look at virtual objects from different viewing angles in the AR interface and perform 3D object manipulation. This approach was implemented using marker-based tracking to improve the tracking in term of the accuracy and robustness in manipulating 3D objects in real-time. We demonstrate our improved tracking system with a sample Tangible AR application, and describe how the system could be improved in the future.

scholarly journals Dimensions of Perception: 3D Real-Life Objects Are More Readily Detected Than Their 2D Images

2021 ◽  
pp. 095679762110107
Author(s):  
Uri Korisky ◽  
Liad Mudrik
Keyword(s):  
Young Adults ◽  
Augmented Reality ◽  
3D Printing ◽  
Real Life ◽  
Interocular Suppression ◽  
Continuous Flash Suppression ◽  
3D Objects ◽  
Real Objects ◽  
2D Images

Most of our interactions with our environment involve manipulating real 3D objects. Accordingly, 3D objects seem to enjoy preferential processing compared with 2D images, for example, in capturing attention or being better remembered. But are they also more readily perceived? Thus far, the possibility of preferred detection for real 3D objects could not be empirically tested because suppression from awareness has been applied only to on-screen stimuli. Here, using a variant of continuous flash suppression (CFS) with augmented-reality goggles (“real-life” CFS), we managed to suppress both real 3D objects and their 2D representations. In 20 healthy young adults, real objects broke suppression faster than their photographs. Using 3D printing, we also showed in 50 healthy young adults that this finding held only for meaningful objects, whereas no difference was found for meaningless, novel ones (a similar trend was observed in another experiment with 20 subjects, yet it did not reach significance). This suggests that the effect might be mediated by affordances facilitating detection of 3D objects under interocular suppression.

scholarly journals Development of a Real-Time Detection System for Augmented Reality Driving

2015 ◽  
Vol 2015 ◽  
pp. 1-5 ◽  
Author(s):  
Kuei-Shu Hsu ◽  
Chia-Sui Wang ◽  
Jinn-Feng Jiang ◽  
Hung-Yuan Wei
Keyword(s):  
Virtual Reality ◽  
Augmented Reality ◽  
Real Time ◽  
Detection System ◽  
Real Life ◽  
Error Rates ◽  
Interactive Effects ◽  
Gaming Technology ◽  
Augmented Reality Technology ◽  
Real Scenario

Augmented reality technology is applied so that driving tests may be performed in various environments using a virtual reality scenario with the ultimate goal of improving visual and interactive effects of simulated drivers. Environmental conditions simulating a real scenario are created using an augmented reality structure, which guarantees the test taker’s security since they are not subject to real-life elements and dangers. Furthermore, the accuracy of tests conducted through virtual reality is not influenced by either environmental or human factors. Driver posture is captured in real time using Kinect’s depth perception function and then applied to driving simulation effects that are emulated by Unity3D’s gaming technology. Subsequently, different driving models may be collected through different drivers. In this research, nearly true and realistic street environments are simulated to evaluate driver behavior. A variety of different visual effects are easily available to effectively reduce error rates, thereby significantly improving test security as well as the reliability and reality of this project. Different situation designs are simulated and evaluated to increase development efficiency and build more security verification test platforms using such technology in conjunction with driving tests, vehicle fittings, environmental factors, and so forth.

Remixing real and imaginary in art education with fully immersive virtual reality

2021 ◽  
Vol 17 (3) ◽  
pp. 415-431 ◽  
Author(s):  
Martina Paatela-Nieminen
Keyword(s):  
Virtual Reality ◽  
Student Teachers ◽  
Art Education ◽  
Visual Art ◽  
Real Life ◽  
3D Models ◽  
Immersive Virtual Reality ◽  
3D Objects ◽  
Teachers Experiences ◽  
Digital Material

This article explores digital material/ism by examining student teachers’ experiences, processes and products with fully immersive virtual reality (VR) as part of visual art education. The students created and painted a virtual world, given the name Gretan puutarha (‘Greta’s Garden’), using the Google application Tilt Brush. They also applied photogrammetry techniques to scan 3D objects from the real world in order to create 3D models for their VR world. Additionally, they imported 2D photographs and drawings along with applied animated effects to construct their VR world digitally, thereby remixing elements from real life and fantasy. The students were asked open-ended questions to find out how they created art virtually and the results were analysed using Burdea’s VR concepts of immersion, interaction and imagination. Digital material was created intersubjectively and intermedially while it was also remixed with real and imaginary. Various webs of meanings were created, both intertextual and rhizomatic in nature.

scholarly journals Pengenalan alat musik tradisional Bangka dengan Marker-Based Augmented Reality

2019 ◽  
Vol 5 (2) ◽  
pp. 89 ◽  
Author(s):  
Fransiskus Panca Juniawan ◽  
Dwi Yuny Sylfania ◽  
Harrizki Arie Pradana ◽  
Laurentinus Laurentinus
Keyword(s):  
Augmented Reality ◽  
Light Intensity ◽  
Object Detection ◽  
Real Time ◽  
Performance Test ◽  
3D Models ◽  
Musical Instruments ◽  
Paper Test ◽  
Strongly Agree ◽  
High Level

Dengan berkembangnya teknologi, kesadaran akan pentingnya alat musik tradisional menjadi berkurang. Demikian juga dengan alat musik tradisional Bangka yang mulai kehilangan popularitasnya. Kondisi saat ini, para remaja di Bangka kebanyakan tidak dapat memainkan alat musik tradisionalnya. Begitu juga dengan anak-anak yang belum mengetahui dan bahkan tidak mengenal alat musik tradisional daerah mereka. Jika kondisi ini dibiarkan, dikhawatirkan keberadaan alat musik tradisional Bangka akan hilang, begitu juga dengan sumber daya manusia yang dapat memainkannya. Untuk menghindari hal tersebut, dibuatlah aplikasi pengenalan alat musik tradisional Bangka menggunakan Augmented Reality (AR). AR dipilih karena dapat memberikan gambaran alat musik secara real time dalam bentuk 3D sesuai dengan pergerakan kamera smartphone yang dinamis. Empat objek 3D alat musik yakni dambus, rebab, rebanatamborin, dan gong yang dibuat menggunakan Autodesk Maya. AR yang dibangun menggunakan metode berbasis marker. Metode ini dipilih agar lebih mudah digunakan oleh pengguna yang mayoritasnya adalah anak-anak. Selain itu, kelebihan metode ini memiliki tingkat akurasi posisi yang sangat tinggi. Unity sebagai engine untuk penerapan AR 3D modelling pada sistem Android dan Vuforia SDK sebagai engine pembentuk marker augmented reality. Pengujian fungsional memiliki hasil 100% dengan sistem yang berjalan baik. Hasil pengujian kinerja deteksi objek AR berdasarkan intensitas cahaya diketahui bahwa smartphone yang memiliki dua kamera di bagian belakang dapat mendeteksi objek dengan intensitas cahaya 0 Lux pada malam hari dengan kondisi gelap, sedangkan yang hanya memiliki satu kamera tidak dapat mendeteksi objek. Pengujian warna marker mendapatkan hasil modifikasi warna marker pink, kuning, dan hitam yang masih memungkinkan untuk pendeteksian objek, walaupun objek yang tampil tidak stabil. Dari pengujian kertas marker diketahui bahwa jenis kertas tidak berpengaruh terhadap pendeteksian objek. Pengujian beta dilakukan dengan cara membagikan kuesioner terkait pengalaman pengguna dalam penggunaan sistem. Hasil survei diketahui pengguna merasa sangat setuju dengan nilai sebesar 80%, bahwa penggunaan sistem dapat membantu mereka dalam mengenal alat musik tradisional Bangka.   With the incessant development of technology, awareness on the importance of traditional musical instruments has declined. Similarly, teenagers living in Bangka no longer play their traditional musical instruments, and children are not exposed to their cultural heritage. However, if this continues, it is feared that the existence of traditional Bangka musical instruments will soon go extinct. To avoid this, researchers have proposed an application to identify this media using Augmented Reality (AR). This technique was chosen due to its ability to provide visuals of musical instruments in real time using 3D models in accordance with the dynamic movement of smartphone cameras. This comprises of four 3D objects namely dambus, rebab, rebanatamborin, and gong, which were designed and developed using Autodesk Maya. AR is built using marker-based methods, which was chosen for easy use because majority of its users are children, and its high level of accuracy. Unity was utilized as an engine for its implementation in the Android system, and Vuforia SDK as augmented reality marker-builder engine. Functional testing showed 100% results which means that the system is running well. From the results of the AR object detection performance test based on light intensity it is known that a smartphone with two cameras in the backside has the ability to detect objects with a light intensity of 0 Lux in dark rooms, while the other smartphone with one camera failed to detect the objects. Color testing obtained a modification of marker colors comprising of pink, yellow, and black which are still able to detect objects, although not stable. The paper test marker has no effect on object detection. Beta testing questionnaires were used to obtain information related to user experience. From the survey results, it is known that users strongly agree (80%) that the use of the system helps them to recognize traditional Bangka musical instruments.

scholarly journals Smart Architectural and Urban Heritage: An Applied Reflection

Heritage ◽  
2021 ◽  
Vol 4 (3) ◽  
pp. 2044-2053
Author(s):  
Stefano Brusaporci ◽  
Pamela Maiezza
Keyword(s):  
Augmented Reality ◽  
Multimedia Information ◽  
Real Time ◽  
Case Studies ◽  
Digital Technologies ◽  
3D Models ◽  
Current Configuration ◽  
Urban Heritage ◽  
Mobile Ar

The aim of this paper is to present the use of 3D models and augmented reality (AR) to study and communicate architectural and urban values and, therefore, favor the development of dedicated forms of “smart heritage”. The study rises from a reflection on the concept of “heritage”, as defined in the international documents, intended as an evolving idea that puts together tangible and intangible aspects. Moreover, digital technologies favor “phygital” applications where the digital dimension support the traditional ones. In this way, AR allows the superimposition of multimedia information to heritage, respecting the historical matter of the artefacts, and supporting a “smart heritage” application. In particular, mobile AR, with real-time and ubiquitous visualizations, offers the opportunity to show past urban and architectural configurations to investigate and describe the transformations that have led to the current configuration, and consequently highlighting the present historical and architectural values of the buildings. Two case studies are presented: the square of St. Basilio Monastery, with its historical transformations, and the Basilica of Collemaggio, a pivotal building in the rites of “Perdonanza Celestiniana”.

Sign in / Sign up

Export Citation Format

Share Document