Deep Audio-visual Learning: A Survey

Audio-visual learning, aimed at exploiting the relationship between audio and visual modalities, has drawn considerable attention since deep learning started to be used successfully. Researchers tend to leverage these two modalities to improve the performance of previously considered single-modality tasks or address new challenging problems. In this paper, we provide a comprehensive survey of recent audio-visual learning development. We divide the current audio-visual learning tasks into four different subfields: audio-visual separation and localization, audio-visual correspondence learning, audio-visual generation, and audio-visual representation learning. State-of-the-art methods, as well as the remaining challenges of each subfield, are further discussed. Finally, we summarize the commonly used datasets and challenges.

Factors Limiting Face Matching at Passport Control and in Police Investigations

Face matching entails a comparison between two faces that are unfamiliar to an observer, who must then decide whether these depict the same person or different people. Despite the ubiquity of face matching in practical settings, such as passport control and police investigations, laboratory research has established that this task is highly error-prone, and that many of these errors derive from visual characteristics of to-be-compared face stimuli. Such characteristics include factors such as image quality, lighting, and natural changes in personal appearance, which influence the visual correspondence between face stimuli. In this chapter, factors that are likely to limit face-matching accuracy in real-world settings are reviewed, with the aim of providing insight into how these influence the accuracy of this process and how subsequent errors may be mitigated.

Quality Matters: A New Approach for Detecting Quality Problems in Web Archives

Since the practice of web archiving, or the act of preserving websites as historical, legal, and informational records, become more commonplace in the 2000s, web archives have become valuable sources for historical research. Unfortunately, many archived websites are of low quality and are missing crucial elements. In this paper, we examine the issue of quality and focus on visual correspondence, the similarity in appearance between the original website and its archived counterpart. We examine how the visual correspondence of an archived website can be measured using image similarity measures. Our results indicate that the Structural Similarity Index metric (SSIM) was able to successfully measure visual correspondence. If applied to the Quality Assurance process of an institution, this similarity metric could help web archivists quickly detect quality problems in their web archives, and fix them in order to create high-quality web archives. Depuis que la pratique de l'archivage Web, ou l'acte de préserver les sites Web en tant que documents historiques, juridiques et informatifs, est devenue plus courante dans les années 2000, les archives Web sont devenues des sources précieuses pour la recherche historique. Malheureusement, de nombreux sites Web archivés sont de mauvaise qualité et manquent d'éléments cruciaux. Dans cet article, nous examinons la question de la qualité et nous nous concentrons sur la correspondance visuelle, la similitude d'apparence entre le site Web d'origine et son homologue archivé. Nous examinons comment la correspondance visuelle d'un site Web archivé peut être mesurée à l'aide de mesures de similitude d'image. Nos résultats indiquent que la Structural Similarity Index metric (SSIM) a pu mesurer avec succès la correspondance visuelle. S'il est appliqué au processus d'assurance qualité d'une institution, cette indicateur de similitude pourrait aider les archivistes Web à détecter rapidement les problèmes de qualité dans leurs archives Web et à les résoudre afin de créer des archives Web de haute qualité.

Terrain Analysis in Google Earth Engine: A Method Adapted for High-Performance Global-Scale Analysis

Terrain analysis is an important tool for modeling environmental systems. Aiming to use the cloud-based computing capabilities of Google Earth Engine (GEE), we customized an algorithm for calculating terrain attributes, such as slope, aspect, and curvatures, for different resolution and geographical extents. The calculation method is based on geometry and elevation values estimated within a 3 × 3 spheroidal window, and it does not rely on projected elevation data. Thus, partial derivatives of terrain are calculated considering the great circle distances of reference nodes of the topographic surface. The algorithm was developed using the JavaScript programming interface of the online code editor of GEE and can be loaded as a custom package. The algorithm also provides an additional feature for making the visualization of terrain maps with a dynamic legend scale, which is useful for mapping different extents: from local to global. We compared the consistency of the proposed method with an available but limited terrain analysis tool of GEE, which resulted in a correlation of 0.89 and 0.96 for aspect and slope over a near-global scale, respectively. In addition to this, we compared the slope, aspect, horizontal, and vertical curvature of a reference site (Mount Ararat) to their equivalent attributes estimated on the System for Automated Geospatial Analysis (SAGA), which achieved a correlation between 0.96 and 0.98. The visual correspondence of TAGEE and SAGA confirms its potential for terrain analysis. The proposed algorithm can be useful for making terrain analysis scalable and adapted to customized needs, benefiting from the high-performance interface of GEE.

Visualization of Functional-semantic Information: Integrating the Clause and the Discourse

In this study, we present our experimental design of an interactive interface that allows users to answer linguistically sophisticated queries utilizing functional-semantic information. Building on previous visualizations of linguistic patterning and discourse structure, the proposed visualization interface presents a unified interface for interrogating the functional-semantic structure of arbitrary texts at different levels of details. To evaluate the effectiveness of the interface, we performed a comparative analysis between visualizations of manual gold-standard annotation and those automatically generated by connecting the interface to existing automatic systems, which revealed remarkable visual correspondence between the two when dealing with small to medium texts. A small-scale case study was then conducted which demonstrated the potential of the resulting tool for effective discovery of interesting patterning in large political texts.

From Jerusalemite Spoils to Roman Relics

This chapter, composed in three sections that complete one another, deals with the direct and indirect impact of the nonexistence of the Temple in Jerusalem on the art and architecture in fourth-century Rome. The first section brings together the translation of sacred objects from old (Jewish) and new (Christian) Jerusalem to Rome. The second illustrates how the visual initiative of Dominus legem dat (The Lord gives the Law) was conceivable through the absence of the Temple in Jerusalem and the presence of its relics in Rome. The third section describes the visual correspondence between the scene of Dominus legem dat and the representative Jewish composition of the ark between two menorot, as an outcome of Emperor Julian’s failed attempt to rebuild the Temple in Jerusalem.