Photon-Driven Neural Reconstruction for Path Guiding

Although Monte Carlo path tracing is a simple and effective algorithm to synthesize photo-realistic images, it is often very slow to converge to noise-free results when involving complex global illumination. One of the most successful variance-reduction techniques is path guiding, which can learn better distributions for importance sampling to reduce pixel noise. However, previous methods require a large number of path samples to achieve reliable path guiding. We present a novel neural path guiding approach that can reconstruct high-quality sampling distributions for path guiding from a sparse set of samples, using an offline trained neural network. We leverage photons traced from light sources as the primary input for sampling density reconstruction, which is effective for challenging scenes with strong global illumination. To fully make use of our deep neural network, we partition the scene space into an adaptive hierarchical grid, in which we apply our network to reconstruct high-quality sampling distributions for any local region in the scene. This allows for effective path guiding for arbitrary path bounce at any location in path tracing. We demonstrate that our photon-driven neural path guiding approach can generalize to diverse testing scenes, often achieving better rendering results than previous path guiding approaches and opening up interesting future directions.

Interactive Generation of Path-Traced Lightmaps

<p>Indirect illumination is an important part of realistic images, and accurately simulating the complex effects of indirect illumination in real-time applications has long been a challenge for the industry. One popular approach is to use offline precomputed solutions such as lightmaps (textures containing the precomputed lighting in a scene) to efficiently approximate these effects. Unfortunately, these offline solutions have historically enforced long iteration times that come at a cost to artist productivity. These solutions have additionally either supported only the low-frequency diffuse component of indirect lighting, yielding poor visual results for glossy or metallic materials, or have used overly expensive approximations. In recent years, the state of the art lightmap precomputation pipeline has shifted to using highly vectorised path tracing, often on GPU hardware, to compute the indirect illumination effects. The use of path tracing enables progressive rendering, wherein an approximation to the full solution is found and then refined as opposed to solving for the final result in a single step. Progressive rendering through path tracing thereby helps to provide rapid iteration for artists. This thesis describes a system that can progressively path-trace indirect illumination lightmaps on the GPU.Contributing to this system, itintroduces a new gather-based method for sample accumulation, enhances algorithms from prior work, and presents a range of encoding methods, including a novel progressive method for non-negative least-squares encoding of spherical basis functions. In addition, it presents a novel, efficient solution for high-quality precomputed diffuse and low-frequency specular indirect illumination that extends the Ambient Dice family of spherical basis functions. This solution provides comparable or better specular reconstruction to prior work at lower runtime cost and has potential for widespread use in real-time applications.</p>

Interactive Generation of Path-Traced Lightmaps

<p>Indirect illumination is an important part of realistic images, and accurately simulating the complex effects of indirect illumination in real-time applications has long been a challenge for the industry. One popular approach is to use offline precomputed solutions such as lightmaps (textures containing the precomputed lighting in a scene) to efficiently approximate these effects. Unfortunately, these offline solutions have historically enforced long iteration times that come at a cost to artist productivity. These solutions have additionally either supported only the low-frequency diffuse component of indirect lighting, yielding poor visual results for glossy or metallic materials, or have used overly expensive approximations. In recent years, the state of the art lightmap precomputation pipeline has shifted to using highly vectorised path tracing, often on GPU hardware, to compute the indirect illumination effects. The use of path tracing enables progressive rendering, wherein an approximation to the full solution is found and then refined as opposed to solving for the final result in a single step. Progressive rendering through path tracing thereby helps to provide rapid iteration for artists. This thesis describes a system that can progressively path-trace indirect illumination lightmaps on the GPU.Contributing to this system, itintroduces a new gather-based method for sample accumulation, enhances algorithms from prior work, and presents a range of encoding methods, including a novel progressive method for non-negative least-squares encoding of spherical basis functions. In addition, it presents a novel, efficient solution for high-quality precomputed diffuse and low-frequency specular indirect illumination that extends the Ambient Dice family of spherical basis functions. This solution provides comparable or better specular reconstruction to prior work at lower runtime cost and has potential for widespread use in real-time applications.</p>

Posteriori Reconstruction and Denoising for Path Tracing

<p>Path tracing is a well-established technique for photo-realistic rendering to simulate light path transport. This method has been widely adopted in visual effects industries to generate high quality synthetic images requiring a large number of samples and a long computation time. Due to the high cost to produce the final output, intermediate previsualization of path tracing is in high demand from production artists to detect errors in the early stage of rendering. But visualizing intermediate results of path tracing is also challenging since the synthesized image with limited samples or improper sampling usually suffers from distracting noise. The ideal solution would be to provide a highly plausible intermediate result in the early stages of rendering, using a small fraction of samples, and apply a posteriori manner to approximate the ground truth. In this thesis, this issue is addressed by providing several efficient posteriori reconstructions and denoising technique for previsualization of pa-th tracing. Firstly, we address the problem for the recovery of the missing values to construct low rank matrices for incomplete images including missing pixel, missing sub-pixel, and multi-frame scenarios. A novel approach utilizing a convolutional neural network which provides fast precompletion for initializing missing values, and subsequent weighted nuclear norm minimization with a parameter adjustment strategy efficiently recovers missing values even in high frequency details. The result shows better visual quality compared to the recent methods including compressed sensing based reconstruction. Furthermore, to mitigate the computation budgets of our new approac-h, we extend our method by applying a block Toeplitz structure forming a low-rank matrix for pixel recovery, and tensor structure for multi-frame recovery. In this manner, the reconstruction time can be significantly reduced. Besides that, by exploiting temporal coherence of multi-frame with a tensor structure, we demonstrate an improvement in the overall recovery quality compared to our previous approach. Our recovery methods provide satisfying solution but still require plen-ty of rendering time at prior stage compared with denoising solutions. Finally, we introduce a novel filter for denoising based on convolutional neural network, to address the problem as conventional denoising approach for rendered images. Unlike a plain CNN that applies fixed kernel size in each layer, we propose a multi-scale residual network with various auxiliary scene features to leverage a new efficient denoising filter for path tracing. Our experimental results show on par or better denoising quality compare to state-of-the-art path tracing denoiser.</p>

Posteriori Reconstruction and Denoising for Path Tracing

<p>Path tracing is a well-established technique for photo-realistic rendering to simulate light path transport. This method has been widely adopted in visual effects industries to generate high quality synthetic images requiring a large number of samples and a long computation time. Due to the high cost to produce the final output, intermediate previsualization of path tracing is in high demand from production artists to detect errors in the early stage of rendering. But visualizing intermediate results of path tracing is also challenging since the synthesized image with limited samples or improper sampling usually suffers from distracting noise. The ideal solution would be to provide a highly plausible intermediate result in the early stages of rendering, using a small fraction of samples, and apply a posteriori manner to approximate the ground truth. In this thesis, this issue is addressed by providing several efficient posteriori reconstructions and denoising technique for previsualization of pa-th tracing. Firstly, we address the problem for the recovery of the missing values to construct low rank matrices for incomplete images including missing pixel, missing sub-pixel, and multi-frame scenarios. A novel approach utilizing a convolutional neural network which provides fast precompletion for initializing missing values, and subsequent weighted nuclear norm minimization with a parameter adjustment strategy efficiently recovers missing values even in high frequency details. The result shows better visual quality compared to the recent methods including compressed sensing based reconstruction. Furthermore, to mitigate the computation budgets of our new approac-h, we extend our method by applying a block Toeplitz structure forming a low-rank matrix for pixel recovery, and tensor structure for multi-frame recovery. In this manner, the reconstruction time can be significantly reduced. Besides that, by exploiting temporal coherence of multi-frame with a tensor structure, we demonstrate an improvement in the overall recovery quality compared to our previous approach. Our recovery methods provide satisfying solution but still require plen-ty of rendering time at prior stage compared with denoising solutions. Finally, we introduce a novel filter for denoising based on convolutional neural network, to address the problem as conventional denoising approach for rendered images. Unlike a plain CNN that applies fixed kernel size in each layer, we propose a multi-scale residual network with various auxiliary scene features to leverage a new efficient denoising filter for path tracing. Our experimental results show on par or better denoising quality compare to state-of-the-art path tracing denoiser.</p>

Automatic First Arrival Picking Workflow by Global Path Tracing

It can be challenging to pick high quality first arrivals on noisy seismic datasets. The stability and smoothness criteria of the picked first arrival are not satisfied for datasets with shingles and interferences from unexpected and backscattered events. To improve first arrival picking, we propose an automatic first arrival picking workflow using global path tracing to find a global solution for first arrival picking with the condition of smoothness of the traced path. The proposed methodology is composed of data preconditioning, global path tracing, and final addition of traced and piloted travel times to compute the total picked travel time. We propose several ways to precondition the dataset, including the use of amplitude and amplitude ratio with and without a pilot. 2D global path tracing is comprised of two steps, namely, accumulation of energy on the potential path and backtracking of the optimal path with a strain factor for smoothness. For higher dimensional datasets, two strategies were adopted. One was to split the higher-dimension data into sub-domains of two dimensions to which 2D global path tracing was applied. The alternative method was to smooth the preconditioned dataset in directions except for the one used to trace the path before applying 2D global path tracing. Next, we discussed the importance of choosing proper parameters in both data preconditioning and constraining global path tracing. We demonstrated the robustness and stability of the proposed automatic first arrival picking via global path tracing using synthetic and field data examples.