Big Data-Driven Pedestrian Analytics: Unsupervised Clustering and Relational Query Based on Tencent Street View Photographs

Recent technological advancements in geomatics and mobile sensing have led to various urban big data, such as Tencent street view (TSV) photographs; yet, the urban objects in the big dataset have hitherto been inadequately exploited. This paper aims to propose a pedestrian analytics approach named vectors of uncountable and countable objects for clustering and analysis (VUCCA) for processing 530,000 TSV photographs of Hong Kong Island. First, VUCCA transductively adopts two pre-trained deep models to TSV photographs for extracting pedestrians and surrounding pixels into generalizable semantic vectors of features, including uncountable objects such as vegetation, sky, paved pedestrian path, and guardrail and countable objects such as cars, trucks, pedestrians, city animals, and traffic lights. Then, the extracted pedestrians are semantically clustered using the vectors, e.g., for understanding where they usually stand. Third, pedestrians are semantically indexed using relations and activities (e.g., walking behind a guardrail, road-crossing, carrying a backpack, or walking a pet) for queries of unstructured photographic instances or natural language clauses. The experiment results showed that the pedestrians detected in the TSV photographs were successfully clustered into meaningful groups and indexed by the semantic vectors. The presented VUCCA can enrich eye-level urban features into computational semantic vectors for pedestrians to enable smart city research in urban geography, urban planning, real estate, transportation, conservation, and other disciplines.

Creative Connections: Computational Semantic Distance Captures Individual Creativity and Resting-State Functional Connectivity

Recent studies of creative cognition have revealed interactions between functional brain networks involved in the generation of novel ideas; however, the neural basis of creativity is highly complex and presents a great challenge in the field of cognitive neuroscience, partly because of ambiguity around how to assess creativity. We applied a novel computational method of verbal creativity assessment—semantic distance—and performed weighted degree functional connectivity analyses to explore how individual differences in assembly of resting-state networks are associated with this objective creativity assessment. To measure creative performance, a sample of healthy adults ( n = 175) completed a battery of divergent thinking (DT) tasks, in which they were asked to think of unusual uses for everyday objects. Computational semantic models were applied to calculate the semantic distance between objects and responses to obtain an objective measure of DT performance. All participants underwent resting-state imaging, from which we computed voxel-wise connectivity matrices between all gray matter voxels. A linear regression analysis was applied between DT and weighted degree of the connectivity matrices. Our analysis revealed a significant connectivity decrease in the visual-temporal and parietal regions, in relation to increased levels of DT. Link-level analyses showed higher local connectivity within visual regions was associated with lower DT, whereas projections from the precuneus to the right inferior occipital and temporal cortex were positively associated with DT. Our results demonstrate differential patterns of resting-state connectivity associated with individual creative thinking ability, extending past work using a new application to automatically assess creativity via semantic distance.

Decoding individual identity from brain activity elicited in imagining common experiences

Everyone experiences common events differently. This leads to personal memories that presumably provide neural signatures of individual identity when events are reimagined. We present initial evidence that these signatures can be read from brain activity. To do this, we progress beyond previous work that has deployed generic group-level computational semantic models to distinguish between neural representations of different events, but not revealed interpersonal differences in event representations. We scanned 26 participants’ brain activity using functional Magnetic Resonance Imaging as they vividly imagined themselves personally experiencing 20 common scenarios (e.g., dancing, shopping, wedding). Rather than adopting a one-size-fits-all approach to generically model scenarios, we constructed personal models from participants’ verbal descriptions and self-ratings of sensory/motor/cognitive/spatiotemporal and emotional characteristics of the imagined experiences. We demonstrate that participants’ neural representations are better predicted by their own models than other peoples’. This showcases how neuroimaging and personalized models can quantify individual-differences in imagined experiences.

Neural dynamics of semantic composition

Human speech comprehension is remarkable for its immediacy and rapidity. The listener interprets an incrementally delivered auditory input, millisecond by millisecond as it is heard, in terms of complex multilevel representations of relevant linguistic and nonlinguistic knowledge. Central to this process are the neural computations involved in semantic combination, whereby the meanings of words are combined into more complex representations, as in the combination of a verb and its following direct object (DO) noun (e.g., “eat the apple”). These combinatorial processes form the backbone for incremental interpretation, enabling listeners to integrate the meaning of each word as it is heard into their dynamic interpretation of the current utterance. Focusing on the verb-DO noun relationship in simple spoken sentences, we applied multivariate pattern analysis and computational semantic modeling to source-localized electro/magnetoencephalographic data to map out the specific representational constraints that are constructed as each word is heard, and to determine how these constraints guide the interpretation of subsequent words in the utterance. Comparing context-independent semantic models of the DO noun with contextually constrained noun models reflecting the semantic properties of the preceding verb, we found that only the contextually constrained model showed a significant fit to the brain data. Pattern-based measures of directed connectivity across the left hemisphere language network revealed a continuous information flow among temporal, inferior frontal, and inferior parietal regions, underpinning the verb’s modification of the DO noun’s activated semantics. These results provide a plausible neural substrate for seamless real-time incremental interpretation on the observed millisecond time scales.

Many speakers, many worlds

Quantification (see e.g. Peters and Westerståhl, 2006) is probably one of the most extensively studied phenomena in formal semantics. But because of the specific representation of meaning assumed by model-theoretic semantics (one where a true model of the world is a priori available), research in the area has primarily focused on one question: what is the relation of a quantifier to the truth value of a sentence? In contrast, relatively little has been said about the way the underlying model comes about, and its relation to individual speakers’ conceptual knowledge. In this paper, we make a first step in investigating how native speakers of English model relations between non-grounded sets, by observing how they quantify simple statements. We first give some motivation for our task, from both a theoretical linguistic and computational semantic point of view (§2). We then describe our annotation setup (§3) and follow on with an analysis of the produced dataset, conducting a quantitative evaluation which includes inter-annotator agreement for different classes of predicates (§4). We observe that there is significant agreement between speakers but also noticeable variations. We posit that in set-theoretic terms, there are as many worlds as there are speakers (§5), but the overwhelming use of underspecified quantification in ordinary language covers up the individual differences that might otherwise be observed.

Mapping meaning with distributional methods

The semantics of existential there is discussed in a diachronic, corpus-based perspective. While previous studies of there have been qualitative or relied on interpreting relative frequencies directly, the present study combines multivariate statistical techniques with linguistic theory through distributional semantics. It is argued that existential uses of there in earlier stages of English were not semantically empty, and that the original meaning was primarily deictic rather than locative. This analysis combines key insights from previous studies of existential there with a Construction Grammar perspective, and discusses some methodological concerns regarding statistical methods for creating computational semantic maps from diachronic corpus data.