Laughing Heads: Can Transformers Detect What Makes a Sentence Funny?

The automatic detection of humor poses a grand challenge for natural language processing. Transformer-based systems have recently achieved remarkable results on this task, but they usually (1) were evaluated in setups where serious vs humorous texts came from entirely different sources, and (2) focused on benchmarking performance without providing insights into how the models work. We make progress in both respects by training and analyzing transformer-based humor recognition models on a recently introduced dataset consisting of minimal pairs of aligned sentences, one serious, the other humorous. We find that, although our aligned dataset is much harder than previous datasets, transformer-based models recognize the humorous sentence in an aligned pair with high accuracy (78\%). In a careful error analysis, we characterize easy vs hard instances. Finally, by analyzing attention weights, we obtain important insights into the mechanisms by which transformers recognize humor. Most remarkably, we find clear evidence that one single attention head learns to recognize the words that make a test sentence humorous, even without access to this information at training time.

Assessing the Listening Proficiency of ESL Students in a Pandemic

The debate as to whether or not listening skills can be used as a testing device for measuring language proficiency—particularly grammatical competence—has been quite a controversial issue for the longest time. Yet, current research findings have not been entirely conclusive and sufficient either way. Hence, this particular study was conducted to address the problem given the existing research gap in the field. Firstly, the investigation sought to determine the listening proficiency level of 40 ESL freshmen in a state university for grammatical distinctions utilizing the mean score. Secondly, it identified the learners’ common errors and difficulties encountered in listening for grammatical distinctions using frequency and sentence analysis. To validate the results of the communicative listening test, a writing composition exercise was duly administered. The results showed that the respondents have very good proficiency levels in listening for grammatical distinctions in terms of inflectional patterns of singular and plural forms of subjects or predicates and good proficiency levels in terms of inflectional patterns for present and past tense of verbs. However, it was found that there still exist some problems in terms of more complex syntactic constructions. Results revealed that the respondents were confused and disrupted in some listening items when the test sentence is embedded with intervening words, phrases, or subordinate clauses. This problem may actually be indicative of socio-psycholinguistic factors to retain what they learned in an academic setting, but outside of it, the intervening words easily disrupted their concentration and memory. Lastly, the new normal caused by the COVID 19 pandemic has forced educators and students into a new learning environment: the online classroom. In recognition of this, the test was administered online, and the results similarly processed but with consideration of the electronic platform as a moderating variable. This shift in the modality of instruction affects ways of learning that have not necessarily been plotted before, as observed by EFL and ESL experts like Brown (2000) and Richards (2016). Thus, while it is not the primary focus of the study, this paper nevertheless highlights this new socio-technological aspect as a crucial moderator of listening assessment for better grammatical competence.

Memoria secuencial auditiva y memoria verbal en alumnado con discapacidad intelectual

Introducción. El modelo de Baddeley sobre memoria de trabajo establece la base para identificar los componentes que intervienen en las tareas de repetición inmediata de materiales verbales. Partiendo del mismo, se propone este estudio cuyo objetivo general es analizar la relación entre memoria secuencial auditiva y memoria verbal en alumnado con discapacidad intelectual y, así, proponer una escala que permita comparar, según la edad mental, las puntuaciones de cada sujeto en estas pruebas. Método. Estudio cuantitativo, no experimental, transversal, descriptivo y correlacional. Muestra no probabilística incidental en la que participaron 250 alumnos con discapacidad intelectual (123 alumnas y 127 alumnos), evaluados mediante el subtest de memoria secuencial auditiva del Test Illinois de Aptitudes Psicolingüísticas (ITPA) y las dos listas de oraciones con alta carga de significado y gramatical del Test Sentence Repetition (SentRep). Resultados. Se observan relaciones significativas y positivas entre las listas de mediación semántica y morfosintáctica, y la memoria verbal, e igualmente entre cada una de ellas, la edad mental del sujeto y la memoria secuencial auditiva. Resultados que sustentan la construcción de dos baremos para establecer la edad mental equivalente a la repetición de los dos tipos de oraciones, en memoria verbal y en memoria secuencial auditiva. Discusión/Conclusiones. Se ha demostrado la complementariedad de las pruebas aplicadas y que los baremos construidos permiten comparar las puntuaciones obtenidas por un sujeto en relación a su grupo. No obstante, una propuesta futura es extender la muestra y, de este modo, obtener baremos representativos de la población con discapacidad intelectual que apoyen la formulación de hipótesis diagnósticas sobre el nivel funcional alcanzado por este alumnado en el procesamiento fonológico, sintáctico y semántico.

The effect of augmented speech-language therapy delivered by telerehabilitation on poststroke aphasia—a pilot randomized controlled trial

Objective: Pilot a definitive randomized controlled trial of speech-language telerehabilitation in poststroke aphasia in addition to usual care with regard to recruitment, drop-outs, and language effects. Design: Pilot single-blinded randomized controlled trial. Setting: Telerehabilitation delivered from tertiary rehabilitation center to participants at their home or admitted to secondary rehabilitation centers. Subjects: People with naming impairment due to aphasia following stroke. Intervention: Sixty-two participants randomly allocated to 5 hours of speech and language telerehabilitation by videoconference per week over four consecutive weeks together with usual care or usual care alone. The telerehabilitation targeted functional, expressive language. Main measures: Norwegian Basic Aphasia Assessment: naming (primary outcome), repetition, and auditory comprehension subtests; Verb and Sentence Test sentence production subtest and the Communicative Effectiveness Index at baseline, four weeks, and four months postrandomization. Data were analyzed by intention to treat. Results: No significant between-group differences were seen in naming or auditory comprehension in the Norwegian Basic Aphasia Assessment at four weeks and four months post randomization. The telerehabilitation group ( n = 29) achieved a Norwegian Basic Aphasia Assessment repetition score of 8.9 points higher ( P = 0.026) and a Verb and Sentence Test score 3 points higher ( P = 0.002) than the control group ( n = 27) four months postrandomization. Communicative Effectiveness Index was not significantly different between groups, but increased significantly within both groups. No adverse events were reported. Conclusion: Augmented telerehabilitation via videoconference may be a viable rehabilitation model for aphasia affecting language outcomes poststroke. A definitive trial with 230 participants is needed to confirm results.

Reliability, Validity and Standardization of the Reading Test: Sentence Comprehension

The test called ‘Reading Test: Sentence Comprehension (TELCS)’ has been validated and standardized. Participants (N = 1289, 2nd to 5th grade, 7 to 11-years-old) were stratified in 15 state-schools in Brazil. The TELCS demonstrated reliability and validity to classify reading performance by both school grade and chronological age. Correlations between the TELCS and a General Reading Composite score were high, as were those with reading accuracy rates of word and pseudoword. Cluster analysis suggested a five-class solution: reading disability, below, average, above, and high reading performance. For individual or collective use, TELCS can quickly screen the sentence reading ability, useful to identify those who might need additional support.

Comparison of Frequency Transposition and Frequency Compression for People With Extensive Dead Regions in the Cochlea

The objective was to determine the effects of two frequency-lowering algorithms (frequency transposition, FT, and frequency compression, FC) on audibility, speech identification, and subjective benefit, for people with high-frequency hearing loss and extensive dead regions (DRs) in the cochlea. A single-blind randomized crossover design was used. FT and FC were compared with each other and with a control condition (denoted ‘Control’) without frequency lowering, using hearing aids that were otherwise identical. Data were collected after at least 6 weeks of experience with a condition. Outcome measures were audibility, scores for consonant identification, scores for word-final /s, z/ detection ( S test), sentence-in-noise intelligibility, and a questionnaire assessing self-perceived benefit (Spatial and Qualities of Hearing Scale). Ten adults with steeply sloping high-frequency hearing loss and extensive DRs were tested. FT and FC improved the audibility of some high-frequency sounds for 7 and 9 participants out of 10, respectively. At the group level, performance for FT and FC did not differ significantly from that for Control for any of the outcome measures. However, the pattern of consonant confusions varied across conditions. Bayesian analysis of the confusion matrices revealed a trend for FT to lead to more consistent error patterns than FC and Control. Thus, FT may have the potential to give greater benefit than Control or FC following extended experience or training.

Fine-Grained Emotion Detection in Suicide Notes: A Thresholding Approach to Multi-Label Classification

We present a system to automatically identify emotion-carrying sentences in suicide notes and to detect the specific fine-grained emotion conveyed. With this system, we competed in Track 2 of the 2011 Medical NLP Challenge, 14 where the task was to distinguish between fifteen emotion labels, from guilt, sorrow, and hopelessness to hopefulness and happiness. Since a sentence can be annotated with multiple emotions, we designed a thresholding approach that enables assigning multiple labels to a single instance. We rely on the probability estimates returned by an SVM classifier and experimentally set thresholds on these probabilities. Emotion labels are assigned only if their probability exceeds a certain threshold and if the probability of the sentence being emotion-free is low enough. We show the advantages of this thresholding approach by comparing it to a naïve system that assigns only the most probable label to each test sentence, and to a system trained on emotion-carrying sentences only.