Multi-modal Open World User Identification

User identification is an essential step in creating a personalised long-term interaction with robots. This requires learning the users continuously and incrementally, possibly starting from a state without any known user. In this article, we describe a multi-modal incremental Bayesian network with online learning, which is the first method that can be applied in such scenarios. Face recognition is used as the primary biometric, and it is combined with ancillary information, such as gender, age, height, and time of interaction to improve the recognition. The Multi-modal Long-term User Recognition Dataset is generated to simulate various human-robot interaction (HRI) scenarios and evaluate our approach in comparison to face recognition, soft biometrics, and a state-of-the-art open world recognition method (Extreme Value Machine). The results show that the proposed methods significantly outperform the baselines, with an increase in the identification rate up to 47.9% in open-set and closed-set scenarios, and a significant decrease in long-term recognition performance loss. The proposed models generalise well to new users, provide stability, improve over time, and decrease the bias of face recognition. The models were applied in HRI studies for user recognition, personalised rehabilitation, and customer-oriented service, which showed that they are suitable for long-term HRI in the real world.

S2OSC: A Holistic Semi-Supervised Approach for Open Set Classification

Open set classification (OSC) tackles the problem of determining whether the data are in-class or out-of-class during inference, when only provided with a set of in-class examples at training time. Traditional OSC methods usually train discriminative or generative models with the owned in-class data, and then utilize the pre-trained models to classify test data directly. However, these methods always suffer from the embedding confusion problem, i.e., partial out-of-class instances are mixed with in-class ones of similar semantics, making it difficult to classify. To solve this problem, we unify semi-supervised learning to develop a novel OSC algorithm, S2OSC, which incorporates out-of-class instances filtering and model re-training in a transductive manner. In detail, given a pool of newly coming test data, S2OSC firstly filters the mostly distinct out-of-class instances using the pre-trained model, and annotates super-class for them. Then, S2OSC trains a holistic classification model by combing in-class and out-of-class labeled data with the remaining unlabeled test data in a semi-supervised paradigm. Furthermore, considering that data are usually in the streaming form in real applications, we extend S2OSC into an incremental update framework (I-S2OSC), and adopt a knowledge memory regularization to mitigate the catastrophic forgetting problem in incremental update. Despite the simplicity of proposed models, the experimental results show that S2OSC achieves state-of-the-art performance across a variety of OSC tasks, including 85.4% of F1 on CIFAR-10 with only 300 pseudo-labels. We also demonstrate how S2OSC can be expanded to incremental OSC setting effectively with streaming data.

Elementary Fractal Geometry. 2. Carpets Involving Irrational Rotations

Self-similar sets with the open set condition, the linear objects of fractal geometry, have been considered mainly for crystallographic data. Here we introduce new symmetry classes in the plane, based on rotation by irrational angles. Examples without characteristic directions, with strong connectedness and small complexity, were found in a computer-assisted search. They are surprising since the rotations are given by rational matrices, and the proof of the open set condition usually requires integer data. We develop a classification of self-similar sets by symmetry class and algebraic numbers. Examples are given for various quadratic number fields.

Confronting Deep-Learning and Biodiversity Challenges for Automatic Video-Monitoring of Marine Ecosystems

With the availability of low-cost and efficient digital cameras, ecologists can now survey the world’s biodiversity through image sensors, especially in the previously rather inaccessible marine realm. However, the data rapidly accumulates, and ecologists face a data processing bottleneck. While computer vision has long been used as a tool to speed up image processing, it is only since the breakthrough of deep learning (DL) algorithms that the revolution in the automatic assessment of biodiversity by video recording can be considered. However, current applications of DL models to biodiversity monitoring do not consider some universal rules of biodiversity, especially rules on the distribution of species abundance, species rarity and ecosystem openness. Yet, these rules imply three issues for deep learning applications: the imbalance of long-tail datasets biases the training of DL models; scarce data greatly lessens the performances of DL models for classes with few data. Finally, the open-world issue implies that objects that are absent from the training dataset are incorrectly classified in the application dataset. Promising solutions to these issues are discussed, including data augmentation, data generation, cross-entropy modification, few-shot learning and open set recognition. At a time when biodiversity faces the immense challenges of climate change and the Anthropocene defaunation, stronger collaboration between computer scientists and ecologists is urgently needed to unlock the automatic monitoring of biodiversity.

G -Chain Mixing and G -Chain Transitivity in Metric G-Space

Firstly, we introduce the concept of G -chain mixing, G -mixing, and G -chain transitivity in metric G -space. Secondly, we study their dynamical properties and obtain the following results. (1) If the map f has the G -shadowing property, then the map f is G -chain mixed if and only if the map f is G -mixed. (2) The map f is G -chain transitive if and only if for any positive integer k ≥ 2 , the map f k is G -chain transitive. (3) If the map f is G -pointwise chain recurrent, then the map f is G -chain transitive. (4) If there exists a nonempty open set U satisfying G U = U , U ¯ ≠ X , and f U ¯ ⊂ U , then we have that the map f is not G -chain transitive. These conclusions enrich the theory of G -chain mixing, G -mixing, and G -chain transitivity in metric G -space.

SEALNET: Facial recognition software for ecological studies of harbor seals

Methods for long-term monitoring of coastal species such as harbor seals, are often costly, time-consuming, and highly invasive, underscoring the need for improved techniques for data collection and analysis. Here, we propose the use of automated facial recognition technology for identification of individual seals and demonstrate its utility in ecological and population studies. We created a software package, SealNet, that automates photo identification of seals, using a graphical user interface (GUI) software to identify, align and chip seal faces from photographs and a deep convolutional neural network (CNN) suitable for small datasets (e.g., 100 seals with five photos per seal). We piloted the SealNet technology with a population of harbor seals located within Casco Bay on the coast of Maine, USA. Across two-years of sampling, 2019 and 2020, at seven haul-out sites in Middle Bay, we processed 1529 images representing 408 individual seals and achieved 88% (93%) rank-1 accuracy in closed set (open set) seal identification. We identified four seals that were photographed in both years at neighboring haul-out sites, suggesting that some harbor seals exhibit site fidelity within local bays across years, and that there may be evidence of spatial connectivity among haul-out sites. Using capture-mark-recapture (CMR) calculations, we obtained a rough preliminary population estimate of 4386 seals in the Middle Bay area. SealNet software outperformed a similar face recognition method developed for primates, PrimNet, in identifying seals following training on our seal dataset. The ease and wealth of image data that can be processed using SealNet software contributes a vital tool for ecological and behavioral studies of marine mammals in the emerging field of conservation technology.

Existence of parabolic minimizers to the total variation flow on metric measure spaces

We give an existence proof for variational solutions u associated to the total variation flow. Here, the functions being considered are defined on a metric measure space $$({\mathcal {X}}, d, \mu )$$ ( X , d , μ ) satisfying a doubling condition and supporting a Poincaré inequality. For such parabolic minimizers that coincide with a time-independent Cauchy–Dirichlet datum $$u_0$$ u 0 on the parabolic boundary of a space-time-cylinder $$\Omega \times (0, T)$$ Ω × ( 0 , T ) with $$\Omega \subset {\mathcal {X}}$$ Ω ⊂ X an open set and $$T > 0$$ T > 0 , we prove existence in the weak parabolic function space $$L^1_w(0, T; \mathrm {BV}(\Omega ))$$ L w 1 ( 0 , T ; BV ( Ω ) ) . In this paper, we generalize results from a previous work by Bögelein, Duzaar and Marcellini by introducing a more abstract notion for $$\mathrm {BV}$$ BV -valued parabolic function spaces. We argue completely on a variational level.