MarsExplorer: Exploration of Unknown Terrains via Deep Reinforcement Learning and Procedurally Generated Environments

This paper is an initial endeavor to bridge the gap between powerful Deep Reinforcement Learning methodologies and the problem of exploration/coverage of unknown terrains. Within this scope, MarsExplorer, an openai-gym compatible environment tailored to exploration/coverage of unknown areas, is presented. MarsExplorer translates the original robotics problem into a Reinforcement Learning setup that various off-the-shelf algorithms can tackle. Any learned policy can be straightforwardly applied to a robotic platform without an elaborate simulation model of the robot’s dynamics to apply a different learning/adaptation phase. One of its core features is the controllable multi-dimensional procedural generation of terrains, which is the key for producing policies with strong generalization capabilities. Four different state-of-the-art RL algorithms (A3C, PPO, Rainbow, and SAC) are trained on the MarsExplorer environment, and a proper evaluation of their results compared to the average human-level performance is reported. In the follow-up experimental analysis, the effect of the multi-dimensional difficulty setting on the learning capabilities of the best-performing algorithm (PPO) is analyzed. A milestone result is the generation of an exploration policy that follows the Hilbert curve without providing this information to the environment or rewarding directly or indirectly Hilbert-curve-like trajectories. The experimental analysis is concluded by evaluating PPO learned policy algorithm side-by-side with frontier-based exploration strategies. A study on the performance curves revealed that PPO-based policy was capable of performing adaptive-to-the-unknown-terrain sweeping without leaving expensive-to-revisit areas uncovered, underlying the capability of RL-based methodologies to tackle exploration tasks efficiently.

Decoding intra-tumoral spatial heterogeneity on radiological images using the Hilbert curve

Background Current intra-tumoral heterogeneous feature extraction in radiology is limited to the use of a single slice or the region of interest within a few context-associated slices, and the decoding of intra-tumoral spatial heterogeneity using whole tumor samples is rare. We aim to propose a mathematical model of space-filling curve-based spatial correspondence mapping to interpret intra-tumoral spatial locality and heterogeneity. Methods A Hilbert curve-based approach was employed to decode and visualize intra-tumoral spatial heterogeneity by expanding the tumor volume to a two-dimensional (2D) matrix in voxels while preserving the spatial locality of the neighboring voxels. The proposed method was validated using three-dimensional (3D) volumes constructed from lung nodules from the LIDC-IDRI dataset, regular axial plane images, and 3D blocks. Results Dimensionality reduction of the Hilbert volume with a single regular axial plane image showed a sparse and scattered pixel distribution on the corresponding 2D matrix. However, for 3D blocks and lung tumor inside the volume, the dimensionality reduction to the 2D matrix indicated regular and concentrated squares and rectangles. For classification into benign and malignant masses using lung nodules from the LIDC-IDRI dataset, the Inception-V4 indicated that the Hilbert matrix images improved accuracy (85.54% vs. 73.22%, p < 0.001) compared to the original CT images of the test dataset. Conclusions Our study indicates that Hilbert curve-based spatial correspondence mapping is promising for decoding intra-tumoral spatial heterogeneity of partial or whole tumor samples on radiological images. This spatial-locality-preserving approach for voxel expansion enables existing radiomics and convolution neural networks to filter structured and spatially correlated high-dimensional intra-tumoral heterogeneity.

Hilbert curve vs Hilbert space: exploiting fractal 2D covering to increase tensor network efficiency

We present a novel mapping for studying 2D many-body quantum systems by solving an effective, one-dimensional long-range model in place of the original two-dimensional short-range one. In particular, we address the problem of choosing an efficient mapping from the 2D lattice to a 1D chain that optimally preserves the locality of interactions within the TN structure. By using Matrix Product States (MPS) and Tree Tensor Network (TTN) algorithms, we compute the ground state of the 2D quantum Ising model in transverse field with lattice size up to 64×64, comparing the results obtained from different mappings based on two space-filling curves, the snake curve and the Hilbert curve. We show that the locality-preserving properties of the Hilbert curve leads to a clear improvement of numerical precision, especially for large sizes, and turns out to provide the best performances for the simulation of 2D lattice systems via 1D TN structures.

EPIsHilbert: Prediction of Enhancer-Promoter Interactions via Hilbert Curve Encoding and Transfer Learning

Enhancer-promoter interactions (EPIs) play a significant role in the regulation of gene transcription. However, enhancers may not necessarily interact with the closest promoters, but with distant promoters via chromatin looping. Considering the spatial position relationship between enhancers and their target promoters is important for predicting EPIs. Most existing methods only consider sequence information regardless of spatial information. On the other hand, recent computational methods lack generalization capability across different cell line datasets. In this paper, we propose EPIsHilbert, which uses Hilbert curve encoding and two transfer learning approaches. Hilbert curve encoding can preserve the spatial position information between enhancers and promoters. Additionally, we use visualization techniques to explore important sequence fragments that have a high impact on EPIs and the spatial relationships between them. Transfer learning can improve prediction performance across cell lines. In order to further prove the effectiveness of transfer learning, we analyze the sequence coincidence of different cell lines. Experimental results demonstrate that EPIsHilbert is a state-of-the-art model that is superior to most of the existing methods both in specific cell lines and cross cell lines.

Algorithm for the Conformal 3D Printing on Non-Planar Tessellated Surfaces: Applicability in Patterns and Lattices

In contrast to the traditional 3D printing process, where material is deposited layer-by-layer on horizontal flat surfaces, conformal 3D printing enables users to create structures on non-planar surfaces for different and innovative applications. Translating a 2D pattern to any arbitrary non-planar surface, such as a tessellated one, is challenging because the available software for printing is limited to planar slicing. The present research outlines an easy-to-use mathematical algorithm to project a printing trajectory as a sequence of points through a vector-defined direction on any triangle-tessellated non-planar surface. The algorithm processes the ordered points of the 2D version of the printing trajectory, the tessellated STL files of the target surface, and the projection direction. It then generates the new trajectory lying on the target surface with the G-code instructions for the printer. As a proof of concept, several examples are presented, including a Hilbert curve and lattices printed on curved surfaces, using a conventional fused filament fabrication machine. The algorithm’s effectiveness is further demonstrated by translating a printing trajectory to an analytical surface. The surface is tessellated and fed to the algorithm as an input to compare the results, demonstrating that the error depends on the resolution of the tessellated surface rather than on the algorithm itself.

Modified Hilbert Curve for Rectangles and Cuboids and Its Application in Entropy Coding for Image and Video Compression

In our previous work, by combining the Hilbert scan with the symbol grouping method, efficient run-length-based entropy coding was developed, and high-efficiency image compression algorithms based on the entropy coding were obtained. However, the 2-D Hilbert curves, which are a critical part of the above-mentioned entropy coding, are defined on squares with the side length being the powers of 2, i.e., 2n, while a subband is normally a rectangle of arbitrary sizes. It is not straightforward to modify the Hilbert curve from squares of side lengths of 2n to an arbitrary rectangle. In this short article, we provide the details of constructing the modified 2-D Hilbert curve of arbitrary rectangle sizes. Furthermore, we extend the method from a 2-D rectangle to a 3-D cuboid. The 3-D modified Hilbert curves are used in a novel 3-D transform video compression algorithm that employs the run-length-based entropy coding. Additionally, the modified 2-D and 3-D Hilbert curves introduced in this short article could be useful for some unknown applications in the future.