Unsupervised random forest for affinity estimation

This paper presents an unsupervised clustering random-forest-based metric for affinity estimation in large and high-dimensional data. The criterion used for node splitting during forest construction can handle rank-deficiency when measuring cluster compactness. The binary forest-based metric is extended to continuous metrics by exploiting both the common traversal path and the smallest shared parent node.The proposed forest-based metric efficiently estimates affinity by passing down data pairs in the forest using a limited number of decision trees. A pseudo-leaf-splitting (PLS) algorithm is introduced to account for spatial relationships, which regularizes affinity measures and overcomes inconsistent leaf assign-ments. The random-forest-based metric with PLS facilitates the establishment of consistent and point-wise correspondences. The proposed method has been applied to automatic phrase recognition using color and depth videos and point-wise correspondence. Extensive experiments demonstrate the effectiveness of the proposed method in affinity estimation in a comparison with the state-of-the-art.

Least Squares Adjustment with a Rank-Deficient Weight Matrix and Its Applicability to Image/Lidar Data Processing

This article proposes a solution to special least squares adjustment (LSA) models with a rank-deficient weight matrix, which are commonly encountered in geomatics. The two sources of rank deficiency in weight matrices are discussed: naturally occurring due to the inherent characteristics of LSA mathematical models and artificially induced to eliminate nuisance parameters from LSA estimation. The physical interpretation of the sources of rank deficiency is demonstrated using a case study to solve the problem of 3D line fitting, which is often encountered in geomatics but has not been addressed fully to date. Finally, some geomatics-related applications—mobile lidar system calibration, point cloud registration, and single-photo resection—are discussed along with respective experimental results, to emphasize the need to assess LSA models and their weight matrices to draw inferences regarding the effective contribution of observations. The discussion and results demonstrate the vast applications of this research in geomatics as well as other engineering domains.

$$\ell _2$$-penalized approximate likelihood inference in logit mixed models for regional prevalence estimation under covariate rank-deficiency

Regional prevalence estimation requires the use of suitable statistical methods on epidemiologic data with substantial local detail. Small area estimation with medical treatment records as covariates marks a promising combination for this purpose. However, medical routine data often has strong internal correlation due to diagnosis-related grouping in the records. Depending on the strength of the correlation, the space spanned by the covariates can become rank-deficient. In this case, prevalence estimates suffer from unacceptable uncertainty as the individual contributions of the covariates to the model cannot be identified properly. We propose an area-level logit mixed model for regional prevalence estimation with a new fitting algorithm to solve this problem. We extend the Laplace approximation to the log-likelihood by an $$\ell _2$$ ℓ 2 -penalty in order to stabilize the estimation process in the presence of covariate rank-deficiency. Empirical best predictors under the model and a parametric bootstrap for mean squared error estimation are presented. A Monte Carlo simulation study is conducted to evaluate the properties of our methodology in a controlled environment. We further provide an empirical application where the district-level prevalence of multiple sclerosis in Germany is estimated using health insurance records.

SVD Entropy Reveals the High Complexity of Ecological Networks

Quantifying the complexity of ecological networks has remained elusive. Primarily, complexity has been defined on the basis of the structural (or behavioural) complexity of the system. These definitions ignore the notion of “physical complexity,” which can measure the amount of information contained in an ecological network, and how difficult it would be to compress. We present relative rank deficiency and SVD entropy as measures of “external” and “internal” complexity, respectively. Using bipartite ecological networks, we find that they all show a very high, almost maximal, physical complexity. Pollination networks, in particular, are more complex when compared to other types of interactions. In addition, we find that SVD entropy relates to other structural measures of complexity (nestedness, connectance, and spectral radius), but does not inform about the resilience of a network when using simulated extinction cascades, which has previously been reported for structural measures of complexity. We argue that SVD entropy provides a fundamentally more “correct” measure of network complexity and should be added to the toolkit of descriptors of ecological networks moving forward.

An Efficient DOA Estimation Method for Passive Surveillance System Based on Troposcatter

Direction of arrival (DOA) estimation plays an important role in the passive surveillance system based on troposcatter. Rank deficiency and subspace leakage resulting from multipath propagation can deteriorate the performance of the DOA estimator. In this paper, characteristics of signals propagated by troposcatter are analyzed, and an efficient DOA estimation method is proposed. According to our new method, the invariance property of noise subspace (IPNS) is introduced as the main method. To provide precise noise subspace for INPS, forward and backward spatial smoothing (FBSS) is carried out to overcome rank deficiency. Subspace leakage is eliminated by a two-step scheme, and this process can also largely reduce the computational load of INPS. Numerical simulation results validate that our method has not only good resolution in condition of closely spaced signals but also superior performance in case of power difference.

The Use of the Lattice Packing Theory for Precise Point Positioning with Ionosphere-Free Measurements in CDMA GNSS

The paper considers the use of the lattice packing theory for Integer Precise Point Positioning (Integer PPP) with the errors usually not exceeding 1–3 cm based on GNSS signals with code division multiple access (CDMA). Positioning is carried out by processing ionosphere-free linear combinations of code and phase measurements with ambiguity resolution employing satellite corrections. The main issue of PPP algorithms is overcoming the rank deficiency problem of the linear equation system obtained by linearization of nonlinear mathematical models of measurements. Nowadays Float PPP is quite well developed, where rank deficiency is tackled by combining systematic biases in measurement models with integer carrier phase ambiguities. As a result, the number of unknowns is reduced to the rank of design matrix, which allows unambiguous estimation of precise user coordinates and values of new variables generated by the performed combinations. However, under such conditions the information about integer nature of carrier phase ambiguities is lost, and this leads to a significant increase in convergence time to obtain user coordinates estimates with the errors of 1–3 cm. It is possible to involve the information on the integer nature of phase ambiguities into processing by applying ambiguity resolution algorithms. Though, as a result of the conducted combinations, the integer nature is destroyed, which makes it impossible to apply these algorithms. In Integer PPP rank deficiency is overcome by projecting the state space of the initial linear equation system onto a so-called S-space, whose dimension is equal to the rank of this system. The orientation of the S-space and the direction of projecting are chosen so that the variables of the initial system corresponding to user coordinates are not changed during the projecting and the projections of integer variables remain integer. This makes it possible to estimate precise user coordinates involving information on the integer nature of phase ambiguities. In the literature on Integer PPP based on CDMA GNSS signals processing the description of the S-space orientation with the desired properties is given, but there is no description of the method to determine this orientation. This paper based on the notions of the lattice packing theory considers an algorithm for determining the S-space with the desired properties. It is shown that there exists an infinite set of such S-spaces connected by unimodular transformations, and a technique is proposed to enable selection from this set the S-space, which requires minimal computational cost. The use of the lattice packing theory to the Integer PPP network solution with CDMA GNSS signals will be considered in the following publication of the authors.

SVD entropy reveals the high complexity of ecological networks

Quantifying the complexity of ecological networks has remained elusive. Primarily, complexity has been defined on the basis of the structural (or behavioural) complexity of the system. These definitions ignore the notion of 'physical complexity', which can measure the amount of information contained in an ecological network, and how difficult it would be to compress. We present relative rank deficiency and SVD entropy as measures of 'external' and 'internal' complexity respectively. Using bipartite ecological networks, we find that they all show a very high, almost maximal, physical complexity. Pollination networks, in particular, are more complex when compared to other types of interactions. In addition, we find that SVD entropy relates to other structural measures of complexity (nestedness, connectance, and spectral radius), but does not inform about the resilience of a network when using simulated extinction cascades, which has previously been reported for structural measures of complexity. We argue that SVD entropy provides a fundamentally more 'correct' measure of network complexity and should be added to the toolkit of descriptors of ecological networks moving forward.