Removing Foreground Occlusions in Light Field using Micro-lens Dynamic Filter

Foreground occlusion removal task aims to automatically detect and remove foreground occlusions and recover background objects. Since for Light Fields (LFs), background objects occluded in some views may be seen in other views, the foreground occlusion removal task for LFs is easy to achieve. In this paper, we propose a learning-based method combining ‘seeking’ and ‘generating’ to recover occluded background. Specifically, the micro-lens dynamic filters are proposed to ‘seek’ occluded background points in shifted micro-lens images and remove occlusions using angular information. The shifted images are then combined to further ‘generate’ background regions to supplement more background details using spatial information. By fully exploring the angular and spatial information in LFs, the dense and complex occlusions can be easily removed. Quantitative and qualitative experimental results show that our method outperforms other state-of-the-arts methods by a large margin.

Download Full-text

DEPTH MAP ESTIMATION IN LIGHT FIELDS USING AN STEREO-LIKE TAXONOMY

Revista de Investigaciones Universidad del Quindío ◽

10.33975/riuq.vol28n1.37 ◽

2016 ◽

Vol 28 (1) ◽

pp. 92-100

Author(s):

Francisco C. Calderon ◽

Carlos A. Parra ◽

Cesar L. Niño

Keyword(s):

Light Field ◽

Spatial Information ◽

Depth Map ◽

Ground Truth ◽

Light Fields ◽

Single Camera ◽

Matching Cost ◽

Winner Takes All ◽

The One ◽

The Cost

The light field or LF is a function that describes the amount of light traveling in every direction (angular) through every point (spatial) in a scene, this LF can be captured in several ways, using arrays of cameras, or more recently using a single camera with an special lens, that allows the capture of angular and spatial information of light rays of a scene (LF). This recent camera implementation gives a different approach to find the dept of a scene using only a single camera. In order to estimate the depth, we describe a taxonomy, similar to the one used in stereo Depth-map algorithms. That consist in the creation of a cost tensor to represent the matching cost between different disparities, then, using a support weight window, aggregate the cost tensor, finally, using a winner-takes-all optimization algorithm, search for the best disparities. This paper explains in detail the several changes made to an stereo-like taxonomy, to be applied in a light field, and evaluate this algorithm using a recent database that for the first time, provides several ground-truth light fields, with a respective ground-truth depth map.

Download Full-text

EANet: Depth Estimation Based on EPI of Light Field

BioMed Research International ◽

10.1155/2021/8293151 ◽

2021 ◽

Vol 2021 ◽

pp. 1-10

Author(s):

Yunzhang Du ◽

Qian Zhang ◽

Dingkang Hua ◽

Jiaqi Hou ◽

Bin Wang ◽

...

Keyword(s):

Neural Network ◽

Medical Treatment ◽

Light Field ◽

Spatial Information ◽

Depth Map ◽

Depth Estimation ◽

Experimental Results ◽

Depth Information ◽

The Neural Network ◽

Field Information

The light field is an important way to record the spatial information of the target scene. The purpose of this paper is to obtain depth information through the processing of light field information and provide a basis for intelligent medical treatment. In this paper, we first design an attention module to extract the features of light field images and connect all the features as a feature map to generate an attention image. Then, the attention map is integrated with the convolution layer in the neural network in the form of weights to enhance the weight of the subaperture viewpoint, which is more meaningful for depth estimation. Finally, the obtained initial depth results were optimized. The experimental results show that the MSE, PSNR, and SSIM of the depth map obtained by this method are increased by about 13%, 10 dB, and 4%, respectively, in some scenarios with good performance.

Download Full-text

Micro-lens Image Stack Upsampling for Densely-Sampled Light Field Reconstruction

IEEE Transactions on Computational Imaging ◽

10.1109/tci.2021.3099636 ◽

2021 ◽

pp. 1-1

Author(s):

Shuo Zhang ◽

Song Chang ◽

Zeqi Shen ◽

Youfang Lin

Keyword(s):

Light Field ◽

Image Stack ◽

Field Reconstruction ◽

Lens Image ◽

Micro Lens

Download Full-text

Freezing Technology: Challenges and Prospects for Sustainable Development in Urban Infrastructure

Journal of the Civil Engineering Forum ◽

10.22146/jcef.43945 ◽

2019 ◽

Vol 5 (2) ◽

pp. 169

Author(s):

Shunji Kanie

Keyword(s):

Sustainable Development ◽

Urban Infrastructure ◽

Experimental Results ◽

Practical Application ◽

Environmental Friendliness ◽

Use Of Space ◽

Freezing Soil ◽

The Arts ◽

Ground Freezing ◽

Frozen Sand

Ground freezing has been broadly applied to construction and maintenance works of infrastructures because of its environmental friendliness. Since freezing technology represented by ground freezing can improve the strength of soil as well as its water-tightness, it becomes an essential technology for construction and maintenance of urban infrastructures where the use of space in underground has already been highly integrated. In this paper, overview of the freezing technology is introduced with some important characteristics of freezing soil for practical application. In addition, freezing technology is used for interesting works which could not be completed without freezing, and the state of the arts in freezing technology is presented. A pipe-in-pipe, now the authors are developing, is an example to utilize the potential of frozen sand, and the effect of freezing is explained with experimental results.

Download Full-text

Review of light field technologies

Visual Computing for Industry Biomedicine and Art ◽

10.1186/s42492-021-00096-8 ◽

2021 ◽

Vol 4 (1) ◽

Author(s):

Shuyao Zhou ◽

Tianqian Zhu ◽

Kanle Shi ◽

Yazi Li ◽

Wen Zheng ◽

...

Keyword(s):

Light Field ◽

Depth Estimation ◽

Industrial Applications ◽

Light Fields ◽

Physical Concept ◽

Network Bandwidth ◽

The Past ◽

Art Research ◽

Near Future ◽

Field Imaging

AbstractLight fields are vector functions that map the geometry of light rays to the corresponding plenoptic attributes. They describe the holographic information of scenes by representing the amount of light flowing in every direction through every point in space. The physical concept of light fields was first proposed in 1936, and light fields are becoming increasingly important in the field of computer graphics, especially with the fast growth of computing capacity as well as network bandwidth. In this article, light field imaging is reviewed from the following aspects with an emphasis on the achievements of the past five years: (1) depth estimation, (2) content editing, (3) image quality, (4) scene reconstruction and view synthesis, and (5) industrial products because the technologies of lights fields also intersect with industrial applications. State-of-the-art research has focused on light field acquisition, manipulation, and display. In addition, the research has extended from the laboratory to industry. According to these achievements and challenges, in the near future, the applications of light fields could offer more portability, accessibility, compatibility, and ability to visualize the world.

Download Full-text

Hyperspectral Pansharpening Based on Homomorphic Filtering and Weighted Tensor Matrix

Remote Sensing ◽

10.3390/rs11091005 ◽

2019 ◽

Vol 11 (9) ◽

pp. 1005

Author(s):

Jiahui Qu ◽

Yunsong Li ◽

Qian Du ◽

Wenqian Dong ◽

Bobo Xi

Keyword(s):

Spatial Resolution ◽

Spatial Information ◽

Mean Squared Error ◽

High Spatial Resolution ◽

Experimental Results ◽

Spectral Information ◽

Morphological Operation ◽

Effective Technique ◽

Squared Error ◽

Homomorphic Filtering

Hyperspectral pansharpening is an effective technique to obtain a high spatial resolution hyperspectral (HS) image. In this paper, a new hyperspectral pansharpening algorithm based on homomorphic filtering and weighted tensor matrix (HFWT) is proposed. In the proposed HFWT method, open-closing morphological operation is utilized to remove the noise of the HS image, and homomorphic filtering is introduced to extract the spatial details of each band in the denoised HS image. More importantly, a weighted root mean squared error-based method is proposed to obtain the total spatial information of the HS image, and an optimized weighted tensor matrix based strategy is presented to integrate spatial information of the HS image with spatial information of the panchromatic (PAN) image. With the appropriate integrated spatial details injection, the fused HS image is generated by constructing the suitable gain matrix. Experimental results over both simulated and real datasets demonstrate that the proposed HFWT method effectively generates the fused HS image with high spatial resolution while maintaining the spectral information of the original low spatial resolution HS image.

Download Full-text

Miniaturized 3D Depth Sensing-Based Smartphone Light Field Camera

Sensors ◽

10.3390/s20072129 ◽

2020 ◽

Vol 20 (7) ◽

pp. 2129 ◽

Cited By ~ 1

Author(s):

Hyun Myung Kim ◽

Min Seok Kim ◽

Gil Ju Lee ◽

Hyuk Jae Jang ◽

Young Min Song

Keyword(s):

Light Field ◽

Optical Simulation ◽

Depth Camera ◽

Optical Parameters ◽

Depth Sensing ◽

Camera Systems ◽

Wide Range ◽

3D Camera ◽

Single Aperture ◽

Micro Lens

The miniaturization of 3D depth camera systems to reduce cost and power consumption is essential for their application in electrical devices that are trending toward smaller sizes (such as smartphones and unmanned aerial systems) and in other applications that cannot be realized via conventional approaches. Currently, equipment exists for a wide range of depth-sensing devices, including stereo vision, structured light, and time-of-flight. This paper reports on a miniaturized 3D depth camera based on a light field camera (LFC) configured with a single aperture and a micro-lens array (MLA). The single aperture and each micro-lens of the MLA serve as multi-camera systems for 3D surface imaging. To overcome the optical alignment challenge in the miniaturized LFC system, the MLA was designed to focus by attaching it to an image sensor. Theoretical analysis of the optical parameters was performed using optical simulation based on Monte Carlo ray tracing to find the valid optical parameters for miniaturized 3D camera systems. Moreover, we demonstrated multi-viewpoint image acquisition via a miniaturized 3D camera module integrated into a smartphone.

Download Full-text

Integration of Convolutional Neural Network and Error Correction for Indoor Positioning

ISPRS International Journal of Geo-Information ◽

10.3390/ijgi9020074 ◽

2020 ◽

Vol 9 (2) ◽

pp. 74

Author(s):

Eric Hsueh-Chan Lu ◽

Jing-Mei Ciou

Keyword(s):

Neural Network ◽

Convolutional Neural Network ◽

Error Correction ◽

Spatial Information ◽

Rapid Development ◽

Indoor Positioning ◽

Experimental Results ◽

Global Navigation Satellite Systems ◽

Signal Interference ◽

Position Vector

With the rapid development of surveying and spatial information technologies, more and more attention has been given to positioning. In outdoor environments, people can easily obtain positioning services through global navigation satellite systems (GNSS). In indoor environments, the GNSS signal is often lost, while other positioning problems, such as dead reckoning and wireless signals, will face accumulated errors and signal interference. Therefore, this research uses images to realize a positioning service. The main concept of this work is to establish a model for an indoor field image and its coordinate information and to judge its position by image eigenvalue matching. Based on the architecture of PoseNet, the image is input into a 23-layer convolutional neural network according to various sizes to train end-to-end location identification tasks, and the three-dimensional position vector of the camera is regressed. The experimental data are taken from the underground parking lot and the Palace Museum. The preliminary experimental results show that this new method designed by us can effectively improve the accuracy of indoor positioning by about 20% to 30%. In addition, this paper also discusses other architectures, field sizes, camera parameters, and error corrections for this neural network system. The preliminary experimental results show that the angle error correction method designed by us can effectively improve positioning by about 20%.

Download Full-text