Syntactic Configuration of Code-Switching between Indonesian and English: Another Perspective on Code-Switching Phenomena

Code-switching, an alternation or mixing one language with another, has been an unmarked phenomenon for a multilingual society. In Indonesia, this phenomenon nowadays lives and thrives among the people. This study discusses the syntactic configuration of code-switching between Indonesian and English in terms of switched segments, points, and changing types. The study is descriptive qualitative in nature. The data comprise 25 recording hours of natural speech produced by 119 Indonesians in 4 types of interaction: seminars, meetings, TV dialogues, and chitchats conducted in six metropolitan cities—Jakarta, Bandung, Semarang, Yogyakarta, Surabaya, and Makassar. The sample drawn purposively comprises 550 switching discourses consisting of 666 switching corpora. It is found that nouns serving as subjects, predicators, objects of verbs, and prepositions to be the most dominant switched segments. A switch between Indonesian noun phrases and English noun phrases, Indonesian verbs or prepositions, and English objective noun phrases, Indonesian conjunctions, and English conjoined noun phrases or clauses is the most popular switched points, and intercausal switching including intraporal and interlexical switching is the most frequent switching type of code-switching between Indonesian and English. ANOVA Friedman’s test confirms that these patterns are the same among the four types of discourses, implying that such a syntactic configuration of Indonesia-English code-switching is universally applicable to any situation and type of interaction. In conclusion, the domination of nouns indicates that the syntactic configuration of Indonesian-English code-switching mainly occurs at minor constituents such as within a clause, phrase, and word boundaries. This demonstrates that code-switching between Indonesian and English is more likely to occur intrasentential rather than intersentential, which is the most popular anywhere in literature.

The channel capacity of multilevel linguistic features constrains speech comprehension

Humans are expert at processing speech but how this feat is accomplished remains a major question in cognitive neuroscience. Capitalizing on the concept of channel capacity, we developed a unified measurement framework to investigate the respective influence of seven acoustic and linguistic features on speech comprehension, encompassing acoustic, sub-lexical, lexical and supra-lexical levels of description. We show that comprehension is independently impacted by all these features, but at varying degrees and with a clear dominance of the syllabic rate. Comparing comprehension of French words and sentences further reveals that when supra-lexical contextual information is present, the impact of all other features is dramatically reduced. Finally, we estimated the channel capacity associated with each linguistic feature and compared them with their generic distribution in natural speech. Our data point towards supra-lexical contextual information as the feature limiting the flow of natural speech. Overall, this study reveals how multilevel linguistic features constrain speech comprehension.

Gesture, Prosody and Information Structure Synchronisation in Turkish

<p>This thesis investigates the synchronisation of gesture with prosody and information structure in Turkish. Speech and gesture have a close relationship in human communication, and they are tightly coordinated in production. Research has shown that gestural units are synchronised with prosodic units on a prominence-related micro level (i.e., pitch accents and gesture apexes), however these studies have largely been on a small number of languages of a similar prosodic type, not including Turkish, which has prominence-less prosodic words. It is known that both gesture and speech, through prosody, are hierarchically structured with nested phrasal constituents, but little is known about gesture-prosody synchronisation at this macro level. Even less is known about the timing relationships of gesture with information structure, which is also closely related to prosody. This thesis links gesture to information structure as a part of a three-way synchronisation relationship of gesture, prosody, and information structure. Four participants were filmed in a narrative task, resulting in three hours of Turkish natural speech and gesture data. Selected sections were annotated for prosody using an adapted scheme for Turkish in the Autosegmental-Metrical framework, for information structure and for gesture. In total, there were over 20,000 annotations. The synchronisation of gesture and speech units was systematically investigated at (1) the micro level, and (2) the macro level. At the micro level, this thesis asked which tones apexes are synchronised with, and whether this synchronisation depends on other prosodic and gestural features. It was found that gesture apexes were synchronised with pitch accents if there were pitch accents in the relevant prosodic phrases; if not, they were synchronised with low tones that marked the onsets of prosodic words. This synchronisation pattern was largely consistent across different prosodic and gestural contexts, although it was tighter in the nuclear area. These findings confirm prominence as a constraint on synchronisation with evidence of pitch accent-apex synchronisation. The findings also extend our knowledge of the typology of micro-level synchronisation to cases where prominence is locally absent showing that micro-level synchronisation also obeys the prosodic hierarchy. At the macro level, the aim was to find the prosodic anchor for single gesture phrases while testing for the possible effects of prosodic, gestural and information structural contexts. The findings showed that there was no one-to-one synchronisation of single gesture phrases with single intermediate or intonational phrases. However, it was found that gesture phrases often spanned over multiple consecutive intermediate phrases, and the synchronisation of gesture phrase boundaries was with the boundaries of these intermediate phrase groupings. In addition, these groupings tended to be combinations of pre-nuclear and nuclear intermediate phrases constituting the default focus position in Turkish. This synchronisation behaviour over the focal domain implied that there might be another speech element governing the speech-gesture synchronisation which also informs prosody, i.e. information structure. Based on this finding and a few other associations in the earlier studies, it was hypothesised that gesture is also informed by and synchronised with information structure. In order to test this hypothesis, it was investigated whether gesture phrases were synchronised with information structural units, i.e., topics, foci and background. The findings showed that gesture phrases tended to accompany discursively prominent foci over topics and background. However, gesture phrases did not show perfect synchronisation with any of these information structure units, although there was a systematic overlap in which foci and topics were contained within the duration of complete gesture phrases. Further investigations revealed that gesture phrase parts that bear apex related meaning provided a much better anchor for the synchronisation of information structure units. The preference for accompanying and synchronisation with the parts of gesture bearing gesturally prominent apical meaning also highlighted that prominence is a driving factor of synchronisation at the macro level as well as at the micro level. This thesis has revealed pivotal links between gesture, prosody and information structure through a systematic investigation of synchronisation of these structures. The implications of these links have also been discussed within the thesis, and a model of speech and gesture production integrating synchronisation has been proposed. Overall, the thesis contributes to a deeper understanding of speech and gesture production, explaining how these interact during natural speech.</p>

Gesture, Prosody and Information Structure Synchronisation in Turkish

<p>This thesis investigates the synchronisation of gesture with prosody and information structure in Turkish. Speech and gesture have a close relationship in human communication, and they are tightly coordinated in production. Research has shown that gestural units are synchronised with prosodic units on a prominence-related micro level (i.e., pitch accents and gesture apexes), however these studies have largely been on a small number of languages of a similar prosodic type, not including Turkish, which has prominence-less prosodic words. It is known that both gesture and speech, through prosody, are hierarchically structured with nested phrasal constituents, but little is known about gesture-prosody synchronisation at this macro level. Even less is known about the timing relationships of gesture with information structure, which is also closely related to prosody. This thesis links gesture to information structure as a part of a three-way synchronisation relationship of gesture, prosody, and information structure. Four participants were filmed in a narrative task, resulting in three hours of Turkish natural speech and gesture data. Selected sections were annotated for prosody using an adapted scheme for Turkish in the Autosegmental-Metrical framework, for information structure and for gesture. In total, there were over 20,000 annotations. The synchronisation of gesture and speech units was systematically investigated at (1) the micro level, and (2) the macro level. At the micro level, this thesis asked which tones apexes are synchronised with, and whether this synchronisation depends on other prosodic and gestural features. It was found that gesture apexes were synchronised with pitch accents if there were pitch accents in the relevant prosodic phrases; if not, they were synchronised with low tones that marked the onsets of prosodic words. This synchronisation pattern was largely consistent across different prosodic and gestural contexts, although it was tighter in the nuclear area. These findings confirm prominence as a constraint on synchronisation with evidence of pitch accent-apex synchronisation. The findings also extend our knowledge of the typology of micro-level synchronisation to cases where prominence is locally absent showing that micro-level synchronisation also obeys the prosodic hierarchy. At the macro level, the aim was to find the prosodic anchor for single gesture phrases while testing for the possible effects of prosodic, gestural and information structural contexts. The findings showed that there was no one-to-one synchronisation of single gesture phrases with single intermediate or intonational phrases. However, it was found that gesture phrases often spanned over multiple consecutive intermediate phrases, and the synchronisation of gesture phrase boundaries was with the boundaries of these intermediate phrase groupings. In addition, these groupings tended to be combinations of pre-nuclear and nuclear intermediate phrases constituting the default focus position in Turkish. This synchronisation behaviour over the focal domain implied that there might be another speech element governing the speech-gesture synchronisation which also informs prosody, i.e. information structure. Based on this finding and a few other associations in the earlier studies, it was hypothesised that gesture is also informed by and synchronised with information structure. In order to test this hypothesis, it was investigated whether gesture phrases were synchronised with information structural units, i.e., topics, foci and background. The findings showed that gesture phrases tended to accompany discursively prominent foci over topics and background. However, gesture phrases did not show perfect synchronisation with any of these information structure units, although there was a systematic overlap in which foci and topics were contained within the duration of complete gesture phrases. Further investigations revealed that gesture phrase parts that bear apex related meaning provided a much better anchor for the synchronisation of information structure units. The preference for accompanying and synchronisation with the parts of gesture bearing gesturally prominent apical meaning also highlighted that prominence is a driving factor of synchronisation at the macro level as well as at the micro level. This thesis has revealed pivotal links between gesture, prosody and information structure through a systematic investigation of synchronisation of these structures. The implications of these links have also been discussed within the thesis, and a model of speech and gesture production integrating synchronisation has been proposed. Overall, the thesis contributes to a deeper understanding of speech and gesture production, explaining how these interact during natural speech.</p>

Bilinguals have a single computational system but two compartmentalized phonological grammars: Evidence from code-switching

Classic linguistic models, such as Chomsky’s minimalist schematization of the human language faculty, were typically based on a ‘monolingual ideal’. More recently, models have been extended to bilingual cognition. For instance, MacSwan (2000) posited that bilinguals possess a single syntactic computational system and, crucially, two (or more) phonological grammars. The current paper examines this possible architecture of the bilingual language faculty by using code-switching data, since this type of speech is unique to bilingual and multilingual individuals. Specifically, the natural speech Maria, a habitual Spanish-English code-switcher from the Bangor Miami Corpus, was examined. For the interface of phonology, an analysis was completed on the frequency of syllabic structures used by Maria. Phonotactics were examined as Spanish and English impose differential restrictions on complex onsets and codas. The results indicated that Maria’s language of use impacted the phonotactics of her speech, but that the context of use (unilingual or code-switched) did not. This suggests that Maria was alternating between two independent phonological grammars when she was code-switching. For the interface of morphosyntax, syntactic dependencies within Maria’s code-switched speech and past literature were consulted. The evidence illustrates that syntactic dependencies are indeed established within code-switched sentences, indicating that such constructions are derived from a single syntactic subset. Thus, the quantitative and qualitative results from this paper wholly support MacSwan’s original conjectures regarding the bilingual language faculty: bilingual cognition appears to be composed of a single computational system which builds multi-language syntactic structures, and more than one phonological grammar.

PERCEPTION OF SPEECH USING AUDIO VISUAL AND REPLICA FOR STUDENTS OF SULTAN AGENG TIRTAYASA UNIVERSITY

This paper exmined the perception of speech using audio visual and replica for students of Sultan Ageng Tirtayasa Univesity. This research was aimed at discussing face-to-face conversation or speech felt by the ears and eyes. The prerequisites for audio-visual perception of speech by using ambiguous perceptual sine wave replicas of natural speech as auditory stimuli are studied in details. When the subjects were unaware that auditory stimuli were speech, they only showed a negligible integration of auditory and visual stimuli. The same subjects learn to feel the same auditory stimuli as speech; they integrate auditory and visual stimuli in the same way as natural speech. These research result suggests a special mode of perception of multisensory speech.

Neural tracking as an objective measure of auditory perception and speech intelligibility

The neural tracking framework enables the analysis of neural responses (EEG) to continuous natural speech, e.g., a story or a podcast. This allows for objective investigation of a range of auditory and linguistic processes in the brain during natural speech perception. This approach is more ecologically valid than traditional auditory evoked responses and has great potential for both research and clinical applications. In this article, we review the neural tracking framework and highlight three prominent examples of neural tracking analyses. This includes the neural tracking of the fundamental frequency of the voice (f0), the speech envelope and linguistic features. Each of these analyses provides a unique point of view into the hierarchical stages of speech processing in the human brain. f0-tracking assesses the encoding of fine temporal information in the early stages of the auditory pathway, i.e. from the auditory periphery up to early processing in the primary auditory cortex. This fundamental processing in (mostly) subcortical stages forms the foundation of speech perception in the cortex. Envelope tracking reflects bottom-up and top-down speech-related processes in the auditory cortex, and is likely necessary but not sufficient for speech intelligibility. To study neural processes more directly related to speech intelligibility, neural tracking of linguistic features can be used. This analysis focuses on the encoding of linguistic features (e.g. word or phoneme surprisal) in the brain. Together these analyses form a multi-faceted and time-effective objective assessment of the auditory and linguistic processing of an individual.

Substituting facial movements in singers changes the sounds of musical intervals

Cross-modal integration is ubiquitous within perception and, in humans, the McGurk effect demonstrates that seeing a person articulating speech can change what we hear into a new auditory percept. It remains unclear whether cross-modal integration of sight and sound generalizes to other visible vocal articulations like those made by singers. We surmise that perceptual integrative effects should involve music deeply, since there is ample indeterminacy and variability in its auditory signals. We show that switching videos of sung musical intervals changes systematically the estimated distance between two notes of a musical interval so that pairing the video of a smaller sung interval to a relatively larger auditory led to compression effects on rated intervals, whereas the reverse led to a stretching effect. In addition, after seeing a visually switched video of an equally-tempered sung interval and then hearing the same interval played on the piano, the two intervals were judged often different though they differed only in instrument. These findings reveal spontaneous, cross-modal, integration of vocal sounds and clearly indicate that strong integration of sound and sight can occur beyond the articulations of natural speech.