Social Context-aware Person Search in Videos via Multi-modal Cues

Person search has long been treated as a crucial and challenging task to support deeper insight in personalized summarization and personality discovery. Traditional methods, e.g., person re-identification and face recognition techniques, which profile video characters based on visual information, are often limited by relatively fixed poses or small variation of viewpoints and suffer from more realistic scenes with high motion complexity (e.g., movies). At the same time, long videos such as movies often have logical story lines and are composed of continuously developmental plots. In this situation, different persons usually meet on a specific occasion, in which informative social cues are performed. We notice that these social cues could semantically profile their personality and benefit person search task in two aspects. First, persons with certain relationships usually co-occur in short intervals; in case one of them is easier to be identified, the social relation cues extracted from their co-occurrences could further benefit the identification for the harder ones. Second, social relations could reveal the association between certain scenes and characters (e.g., classmate relationship may only exist among students), which could narrow down candidates into certain persons with a specific relationship. In this way, high-level social relation cues could improve the effectiveness of person search. Along this line, in this article, we propose a social context-aware framework, which fuses visual and social contexts to profile persons in more semantic perspectives and better deal with person search task in complex scenarios. Specifically, we first segment videos into several independent scene units and abstract out social contexts within these scene units. Then, we construct inner-personal links through a graph formulation operation for each scene unit, in which both visual cues and relation cues are considered. Finally, we perform a relation-aware label propagation to identify characters’ occurrences, combining low-level semantic cues (i.e., visual cues) and high-level semantic cues (i.e., relation cues) to further enhance the accuracy. Experiments on real-world datasets validate that our solution outperforms several competitive baselines.

Aspect construal in Mandarin: a usage-based constructionist perspective on LE

Despite extensive research efforts to explain the Mandarin Chinese particle le, confusion persists in the absence of a unitary theory and sufficient empirical evidence. This study provides a unitary account of le by adopting a usage-based constructionist approach, one that liberates grammatical aspect from, and is able to accommodate, lexical aspect. We argue that le participates in two distinct family resemblance constructions of aspect construal associated with two distinct sentential positions. The clause-internal le construction construes the closing or final boundary of an event and the clause-final le construction construes the opening or initial boundary of an event. Corpus analysis showed that the two aspect constructions have distinct patterns in natural language uses that are consistent with the proposed construals. Results from elicited response data showed that native speakers paid attention to construction-level formal and semantic cues in making family resemblance judgments about tokens of the two constructions. This study has both theoretical and methodological implications for crosslinguistic research on grammatical aspect in relation to lexical aspect and for usage-based constructionist approaches to grammatical categories beyond aspect.

Syntactic and semantic interference in sentence comprehension: Support from English and German eye-tracking data

A long-standing debate in the sentence processing literature concerns the time course of syntactic and semantic information in online sentence comprehension. The default assumption in cue-based models of parsing is that syntactic and semantic retrieval cues simultaneously guide dependency resolution. When retrieval cues match multiple items in memory, this leads to similarity-based interference. Both semantic and syntactic interferencehave been shown to occur in English. However, the relative timing of syntactic vs. semantic interference remains unclear. In this first-ever cross-linguistic investigation of the time course of syntactic vs. semantic interference, the data from two eye-tracking reading experiments (English and German) suggest that the two types of interference can in principle arise simultaneously during retrieval. However, the data also indicate that semantic cues may be evaluated with a small timing lag in German compared to English. This suggests that there may be cross-linguistic variation in how syntactic and semantic cues are used to resolve linguistic dependencies in real-time.

Facilitating Word Retrieval in Aphasia: Which Type of Cues for Which Aphasic Speakers?

Background: Even if both phonological and semantic cues can facilitate word retrieval in aphasia, it remains unclear if their respective effectiveness varies according to the underlying anomic profile.Aim: The aim of the present facilitation study is to compare the effect of phonological and semantic cues on picture naming accuracy and speed in different types of anomia.Methods: In the present within-subject design study, 15 aphasic persons following brain damage underwent picture naming paradigms with semantic cues (categorically- or associatively related) and phonological cues (initial phoneme presented auditorily, visually or both).Results: At the group level, semantic cueing was as effective as phonological cueing to significantly speed up picture naming. However, while phonological cues were effective regardless of the anomic profile, semantic cueing effects varied depending on the type of anomia. Participants with mixed anomia showed facilitation after both semantic categorical and associative cues, but individuals with lexical-phonological anomia only after categorical cues. Crucially, semantic cues were ineffective for participants with lexical-semantic anomia. These disparities were confirmed by categorical semantic facilitation decreasing when semantic/omission errors prevailed in the anomic profile, but increasing alongside phonological errors.Conclusion: The effectiveness of phonological vs semantic cues seems related to the underlying anomic profile: phonological cues benefit any type of anomia, but semantic cues only lexical-phonological or mixed anomia.

Speech Perception through Face Masks by Children and Adults

Face masks can cause speech processing difficulties. However, it is unclear to what extent these difficulties are caused by the visual obstruction of the speaker’s mouth or by changes of the acoustic signal, and whether the effects can be found regardless of semantic context. In the present study, children and adults performed a cued shadowing task online, repeating the last word of English sentences. Target words were embedded in sentence-final position and manipulated visually, acoustically, and by semantic context (cloze probability). First results from 16 children and 16 adults suggest that processing language through face masks leads to slower responses in both groups, but visual, acoustic, and semantic cues all significantly reduce the mask effect. Although children were less proficient in predictive speech processing overall, they were still able to use semantic cues to compensate for face mask effects in a similar fashion to adults.

A Rational Model of Incremental Argument Interpretation: The Comprehension of Swedish Transitive Clauses

A central component of sentence understanding is verb-argument interpretation, determining how the referents in the sentence are related to the events or states expressed by the verb. Previous work has found that comprehenders change their argument interpretations incrementally as the sentence unfolds, based on morphosyntactic (e.g., case, agreement), lexico-semantic (e.g., animacy, verb-argument fit), and discourse cues (e.g., givenness). However, it is still unknown whether these cues have a privileged role in language processing, or whether their effects on argument interpretation originate in implicit expectations based on the joint distribution of these cues with argument assignments experienced in previous language input. We compare the former, linguistic account against the latter, expectation-based account, using data from production and comprehension of transitive clauses in Swedish. Based on a large corpus of Swedish, we develop a rational (Bayesian) model of incremental argument interpretation. This model predicts the processing difficulty experienced at different points in the sentence as a function of the Bayesian surprise associated with changes in expectations over possible argument interpretations. We then test the model against reading times from a self-paced reading experiment on Swedish. We find Bayesian surprise to be a significant predictor of reading times, complementing effects of word surprisal. Bayesian surprise also captures the qualitative effects of morpho-syntactic and lexico-semantic cues. Additional model comparisons find that it—with a single degree of freedom—captures much, if not all, of the effects associated with these cues. This suggests that the effects of form- and meaning-based cues to argument interpretation are mediated through expectation-based processing.

Assignment of Grammatical Gender in Heritage Greek

This study investigates the acquisition of grammatical gender in Heritage Greek as acquired by children (6–8 years of age) and adolescents (15–18 years) growing up in Adelaide, South Australia. The determiner elicitation task from Varlokosta (2005) was employed to assess the role of morphological and semantic cues when it comes to gender assignment for real and novel nouns. Ralli’s (1994) inflectional classes for Greek nouns and Anastasiadi-Symeonidi and Cheila-Markopoulou’s (2003) categories of prototypicality were employed in the analysis of the collected data. The performance of heritage speakers was compared to that of monolingual speakers from Greece (Varlokosta, 2011). The results indicate that–beyond age differences in the two groups–a formal phonological rule guides gender assignment in the production of heritage speakers which departs from initial expectations.

Expertise Detection in Crowdsourcing Forums Using the Composition of Latent Topics and Joint Syntactic–Semantic Cues

We develop an NLP method for inferring potential contributors among multitude of users within crowdsourcing forums (CSFs). The method basically provides a way to predict expertise from their structures (syntax–semantic patterns) when crowdsourced votes are unavailable. It primarily deals with tackling core adverse conditions, which hinder the identification of crowds’ expertise levels, and standardization of measuring linguistic quality of crowdsourced text. To solve the former, an expertise estimation and linguistic feature annotation algorithm is developed. To approach the later, a comprehensive linguistic characterization of crowdsourced text, along with extensive joint syntax–punctuation analyses, have been carried out. The entire corpora are comprised of approximately 8 different domains, 3 million and 50,000 sentences, and 32 million and 90,000 words, contributed by a crowd of 50,000 users. The analyses revealed six major linguistic patterns, identified on the basis of ordered lists of structural (syntactic) categories, learned from grammatical constructions, practiced by major groups of experts. In addition, nine different text-oriented expertise dimensions are identified, as crucial steps towards establishing standard linguistic-based expertise-framework for most CSFs. Potentially, the resulting framework simplifies the measurement of crowds’ proficiency, in those particular forums, where crowds’ tasks (e.g., answering questions, technically discerning deep features within images of galaxies for classifying them into certain categories) are intimately connected with their writing (e.g., describing answers illustratively, expressing complex phenomena observed in classified images). Moreover, wide varieties of linguistic annotations: latent topic annotations, named entities, syntactic and punctuation annotations, semantic and character set annotations, word and character n-grams (n = 2 and 3) annotations, are extracted. That is for building baseline and enhanced versions of expertise models (about 20 different models built). The successive achievements of enhancing baseline models, with iteratively adding linguistic feature annotations in a two-stage enhancement process, indicate the adaptability of the learned models.