An Information-Theoretic Measure of Cultural Success

I develop a simple but principled method for measuring the amount of culturally-transmitted information from a written target work that is actually retained in human minds and capable of influencing behavior. Using procedures inspired by Claude Shannon’s 1951 method for estimating the entropy of written English, I estimate the entropy of samples from the target work with a treatment group (those that have read a target work) and a control group (those who are of the same culture but who have not read the target work), using human minds as encoders-decoders in the communication model. KL divergence quantifies the information that the treatment group already knows relative to the control group. This method controls for shared cultural inheritance and does not require commitments to what information from the target work is important. The general technique can be profitably extended to a variety of domains, including evolutionary theory, methods of teaching, and the study of music.

Neighborhood co-occurrence modeling in 3D point cloud segmentation

A significant performance boost has been achieved in point cloud semantic segmentation by utilization of the encoder-decoder architecture and novel convolution operations for point clouds. However, co-occurrence relationships within a local region which can directly influence segmentation results are usually ignored by current works. In this paper, we propose a neighborhood co-occurrence matrix (NCM) to model local co-occurrence relationships in a point cloud. We generate target NCM and prediction NCM from semantic labels and a prediction map respectively. Then, Kullback-Leibler (KL) divergence is used to maximize the similarity between the target and prediction NCMs to learn the co-occurrence relationship. Moreover, for large scenes where the NCMs for a sampled point cloud and the whole scene differ greatly, we introduce a reverse form of KL divergence which can better handle the difference to supervise the prediction NCMs. We integrate our method into an existing backbone and conduct comprehensive experiments on three datasets: Semantic3D for outdoor space segmentation, and S3DIS and ScanNet v2 for indoor scene segmentation. Results indicate that our method can significantly improve upon the backbone and outperform many leading competitors.

Families of Generalized Quasisymmetry Models: A ϕ-Divergence Approach

The quasisymmetry (QS) model for square contingency tables is revisited, highlighting properties and features on the basis of its alternative definitions. More parsimonious QS-type models, such as the ordinal QS model for ordinal classification variables and models based on association models (AMs) with homogeneous row and column scores, are discussed. All these models are linked to the local odds ratios (LOR). QS-type models and AMs were extended in the literature for generalized odds ratios other than LOR. Furthermore, in an information-theoretic context, they are expressed as distance models from a parsimonious reference model (the complete symmetry for QS and the independence for AMs), while they satisfy closeness properties with respect to Kullback–Leibler (KL) divergence. Replacing the KL by ϕ divergence, flexible classes of QS-type models for LOR, AMs for LOR, and AMs for generalized odds ratios were generated. However, special QS-type models that are based on homogeneous AMs for LOR have not been extended to ϕ-divergence-based classes so far, or the QS-type models for generalized odds ratios. In this work, we develop these missing extensions, and discuss QS-type models and their generalizations in depth. These flexible families enrich the modeling options, leading to models of better fit and sound interpretation, as illustrated by representative examples.

Development of an expected possession value model to analyse team attacking performances in rugby league

This study aimed to evaluate team attacking performances in rugby league via expected possession value (EPV) models. Location data from 59,233 plays in 180 Super League matches across the 2019 Super League season were used. Six EPV models were generated using arbitrary zone sizes (EPV-308 and EPV-77) or aggregated according to the total zone value generated during a match (EPV-37, EPV-19, EPV-13 and EPV-9). Attacking sets were considered as Markov Chains, allowing the value of each zone visited to be estimated based on the outcome of the possession. The Kullback-Leibler Divergence was used to evaluate the reproducibility of the value generated from each zone (the reward distribution) by teams between matches. Decreasing the number of zones improved the reproducibility of reward distributions between matches but reduced the variation in zone values. After six previous matches, the subsequent match’s zones had been visited on 95% or more occasions for EPV-19 (95±4%), EPV-13 (100±0%) and EPV-9 (100±0%). The KL Divergence values were infinity (EPV-308), 0.52±0.05 (EPV-77), 0.37±0.03 (EPV-37), 0.20±0.02 (EPV-19), 0.13±0.02 (EPV-13) and 0.10±0.02 (EPV-9). This study supports the use of EPV-19 and EPV-13, but not EPV-9 (too little variation in zone values), to evaluate team attacking performance in rugby league.

Assembly of a Coreset of Earth Observation Images on a Small Quantum Computer

Satellite instruments monitor the Earth’s surface day and night, and, as a result, the size of Earth observation (EO) data is dramatically increasing. Machine Learning (ML) techniques are employed routinely to analyze and process these big EO data, and one well-known ML technique is a Support Vector Machine (SVM). An SVM poses a quadratic programming problem, and quantum computers including quantum annealers (QA) as well as gate-based quantum computers promise to solve an SVM more efficiently than a conventional computer; training the SVM by employing a quantum computer/conventional computer represents a quantum SVM (qSVM)/classical SVM (cSVM) application. However, quantum computers cannot tackle many practical EO problems by using a qSVM due to their very low number of input qubits. Hence, we assembled a coreset (“core of a dataset”) of given EO data for training a weighted SVM on a small quantum computer, a D-Wave quantum annealer with around 5000 input quantum bits. The coreset is a small, representative weighted subset of an original dataset, and its performance can be analyzed by using the proposed weighted SVM on a small quantum computer in contrast to the original dataset. As practical data, we use synthetic data, Iris data, a Hyperspectral Image (HSI) of Indian Pine, and a Polarimetric Synthetic Aperture Radar (PolSAR) image of San Francisco. We measured the closeness between an original dataset and its coreset by employing a Kullback–Leibler (KL) divergence test, and, in addition, we trained a weighted SVM on our coreset data by using both a D-Wave quantum annealer (D-Wave QA) and a conventional computer. Our findings show that the coreset approximates the original dataset with very small KL divergence (smaller is better), and the weighted qSVM even outperforms the weighted cSVM on the coresets for a few instances of our experiments. As a side result (or a by-product result), we also present our KL divergence findings for demonstrating the closeness between our original data (i.e., our synthetic data, Iris data, hyperspectral image, and PolSAR image) and the assembled coreset.

Point Set Registration Based on Improved KL Divergence

A point set registration algorithm based on improved Kullback–Leibler (KL) divergence is proposed. Each point in the point set is represented as a Gaussian distribution. The Gaussian distribution contains the position information of the candidate point and surrounding ones. In this way, the entire point set can be modeled as a Gaussian mixture model (GMM). The registration problem of two point sets is further converted as a minimization problem of the improved KL divergence between two GMMs, and the genetic algorithm is used to optimize the solution. Experimental results show that the proposed algorithm has strong robustness to noise, outliers, and missing points, which achieves better registration accuracy than some state-of-the-art methods.

Methods on Detecting Closely Related Topics and Spatial Events

Topic detection is a hot issue that many researchers are interested in. The previous researches focused on the single data stream, they did not consider the topic detection from different data streams in a harmonious way, so they cannot detect closely related topics from different data streams. Recently, Twitter, along with other SNS such as Weibo, and Yelp, began backing position services in their texts. Previous approaches are either complex to be conducted or oversimplified that cannot achieve better performance on detecting spatial topics. In our paper, we introduce a probabilistic method which can precisely detect closely related bursty topics and their bursty periods across different data streams in a unified way. We also introduce a probabilistic method called Latent Spatial Events Model (LSEM) that can find areas as well as to detect the spatial events, it can also predict positions of the texts. We evaluate LSEM on different datasets and reflect that our approach outperforms other baseline approaches in different indexes such as perplexity, entropy of topic and KL-divergence, range error. Evaluation of our first proposed approach on different datasets shows that it can detect closely related topics and meaningful bursty time periods from different datasets.

KL-MOB: automated COVID-19 recognition using a novel approach based on image enhancement and a modified MobileNet CNN

The emergence of the novel coronavirus pneumonia (COVID-19) pandemic at the end of 2019 led to worldwide chaos. However, the world breathed a sigh of relief when a few countries announced the development of a vaccine and gradually began to distribute it. Nevertheless, the emergence of another wave of this pandemic returned us to the starting point. At present, early detection of infected people is the paramount concern of both specialists and health researchers. This paper proposes a method to detect infected patients through chest x-ray images by using the large dataset available online for COVID-19 (COVIDx), which consists of 2128 X-ray images of COVID-19 cases, 8,066 normal cases, and 5,575 cases of pneumonia. A hybrid algorithm is applied to improve image quality before undertaking neural network training. This algorithm combines two different noise-reduction filters in the image, followed by a contrast enhancement algorithm. To detect COVID-19, we propose a novel convolution neural network (CNN) architecture called KL-MOB (COVID-19 detection network based on the MobileNet structure). The performance of KL-MOB is boosted by adding the Kullback–Leibler (KL) divergence loss function when trained from scratch. The KL divergence loss function is adopted for content-based image retrieval and fine-grained classification to improve the quality of image representation. The results are impressive: the overall benchmark accuracy, sensitivity, specificity, and precision are 98.7%, 98.32%, 98.82% and 98.37%, respectively. These promising results should help other researchers develop innovative methods to aid specialists. The tremendous potential of the method proposed herein can also be used to detect COVID-19 quickly and safely in patients throughout the world.

A Satellite Incipient Fault Detection Method Based on Decomposed Kullback–Leibler Divergence

Detection of faults at the incipient stage is critical to improving the availability and continuity of satellite services. The application of a local optimum projection vector and the Kullback–Leibler (KL) divergence can improve the detection rate of incipient faults. However, this suffers from the problem of high time complexity. We propose decomposing the KL divergence in the original optimization model and applying the property of the generalized Rayleigh quotient to reduce time complexity. Additionally, we establish two distribution models for subfunctions F1(w) and F3(w) to detect the slight anomalous behavior of the mean and covariance. The effectiveness of the proposed method was verified through a numerical simulation case and a real satellite fault case. The results demonstrate the advantages of low computational complexity and high sensitivity to incipient faults.