Spectator View: Enabling Asymmetric Interaction between HMD Wearers and Spectators with a Large Display

In this paper, we present a system that allows a user with a head-mounted display (HMD) to communicate and collaborate with spectators outside of the headset. We evaluate its impact on task performance, immersion, and collaborative interaction. Our solution targets scenarios like live presentations or multi-user collaborative systems, where it is not convenient to develop a VR multiplayer experience and supply each user (and spectator) with an HMD. The spectator views the virtual world on a large-scale tiled video wall and is given the ability to control the orientation of their own virtual camera. This allows spectators to stay focused on the immersed user's point of view or freely look around the environment. To improve collaboration between users, we implemented a pointing system where a spectator can point at objects on the screen, which maps an indicator directly onto the objects in the virtual world. We conducted a user study to investigate the influence of rotational camera decoupling and pointing gestures in the context of HMD-immersed and non-immersed users utilizing a large-scale display. Our results indicate that camera decoupling and pointing positively impacts collaboration. A decoupled view is preferable in situations where both users need to indicate objects of interest in the scene, such as presentations and joint-task scenarios, as it requires a shared reference space. A coupled view, on the other hand, is preferable in synchronous interactions such as remote-assistant scenarios.

Task modality and language-related episodes in young learners: An attempt to manage accuracy and editing

Task modality (oral vs. writing) has been found to affect the production, nature and resolution of the language-related episodes (LREs) produced by adult learners in collaborative interaction, a finding also attested in very recent and still limited research with young learners, a population that deserves greater attention in the literature. Besides, previous research has not yet considered the incorporation of LREs in the final output of both oral and written tasks. Nor has it controlled for the differential levels of accuracy that the oral vs. the written modality demand, or the opportunity for revising the output equally in both modalities. Besides, little is known about learners’ motivation towards tasks of different modality. This article fills these gaps by examining the effect of task-modality on the production of LREs by 10- to 12-year-old schoolchildren performing an oral+writing task and an oral+editing task, as well as its effect on their task motivation. Task modality effects were evinced in terms of nature and incorporation of LREs, the written mode leading to greater focus on form and incorporation of accurately resolved LREs. The possibility of editing the oral output resulted in enhanced target-likeness of resolved LREs. As for task motivation, learners perceived both tasks as equally motivating.

Tiefenbohrungen in den musikwissenschaftlichen Werk- und Autorschaftsbegriff mit digitalen Werkzeugen

The concept of work and authorship have long been the subject of intense reflection in historical musicology. Edition projects deal with composers and their work process and have developed new approaches for this purpose. Against the background of the question of where the ‘text’ to be edited actually ends, this paper discusses the possibilities of making director’s piano score (Regieklavierauszug) digitally visible and thus interpretable (understood as a work ‘sui generis’). Furthermore, the creative process of all participants in the classical sound recording is considered in terms of its digital visibility. On the basis of my research data from the DFG project “The sound recording studio as a discoursive room” at the TU Berlin/Audiokommunikation (participant observation, interviews, transcripts of recording processes, evaluations with MaxQDA) I show that these creative processes in the collaborative interaction of artists and engineers can also be found in classical music. By evaluating these (and other) materials with the help of MEI and TEI, new ways of ‘work’ and ‘authorship’ are also explored in this genre. Ultimately, this should also establish the sound recording as a ‘sui generis’ work.

Collaborative Augmented Digital Twin: A Novel Open-Source Augmented Reality Solution for Training and Maintenance Processes in the Shipyard of the Future

Large companies use a lot of resources on workshop operator training and industrial machinery maintenance since the lack of this practice or its poor implementation increases the cost and risks of operating and handling sensitive and/or hazardous machinery. Industrial Augmented Reality (IAR), a major technology in the Industry 4.0 paradigm that may enhance worker performance, minimize hazards and improve manufacturing processes, could be beneficial in this situation. This paper presents an IAR solution that allows for visualizing and interacting with the digital twin of a critical system. Specifically, the augmented digital twin of an industrial cooler was developed. The proposed IAR system provides a dynamic way to perform operator training with a full-size model of the actual equipment and to provide step-by-step guidance so that maintenance processes can be performed more safely and efficiently. The proposed system also allows several users to use devices at the same time, creating a new type of collaborative interaction by viewing the model in the same place and state. Performance tests with many simultaneous users have been conducted, with response latency being measured as the number of connected users grows. Furthermore, the suggested IAR system has been thoroughly tested in a real-world industrial environment.

The Effect of Applying Robot-Assisted Task-Oriented Training Using Human-Robot Collaborative Interaction Force Control Technology on Upper Limb Function in Stroke Patients: Preliminary Findings

Stroke is one of the leading causes of death and the primary cause of acquired disability worldwide. Many stroke survivors have difficulty using their upper limbs, which have important functional roles in the performance of daily life activities. Consequently, the independence and quality of life of most stroke patients are reduced. Robot-assisted therapy is an effective intervention for improving the upper limb function of individuals with stroke. Human-robot collaborative interaction force control technology is critical for improving the flexibility and followability of the robot’s motion, thereby improving rehabilitation training outcomes. However, there are few reports on the effect of robot-assisted rehabilitative training on upper limb function. We applied this technology using a robot to assist patients with task-oriented training. Posttreatment changes in Fugl-Meyer and modified Barthel index (MBI) scores were assessed to determine whether this technology could improve the upper limb function of stroke patients. One healthy adult and five stroke patients, respectively, participated in functional and clinical experiments. The MBI and Fugl-Meyer scores of the five patients in the clinical experiments showed significant improvements after the intervention. The experimental results indicate that human-robot collaborative interaction force control technology is valuable for improving robots’ properties and patients’ recovery. This trial was registered in the Chinese clinical trial registry (ChiCTR2000038676).

Interaction patterns and support for learning in the primary foreign language classroom

In this study, both sociocultural and cognitive perspectives are used to investigate how learners in the primary foreign language classroom support each other’s learning during peer oral interaction, and how this is influenced by different interaction patterns. Learners were recorded taking part in 3 spot-the-difference tasks in a year 4 primary class, and Storch’s model of interaction patterns (2002) was used as a framework to classify learner interaction. Transcripts were analysed qualitatively and quantitatively, and show how learners used a variety of strategies to support their partner’s output. It also shows how the majority of learners worked collaboratively, how collaboration increased with task repetition, and how pairs who engaged in collaborative interaction provided most support for their peer. However, other dyads showed less mutuality and engaged little with each other’s contributions, with quantitative analysis showing these learners provided each other with the least support for language learning, as they were unlikely to ask their peer for help, one of the most common strategies used by other dyads.

Cooperation in sound and motion: Complexity matching in collaborative interaction

Complex behaviors are layered with processes across timescales that must be coordinated with each other to accomplish cooperative goals. Complexity matching is the coordination of nested layers of behaviors across individuals. We hypothesize that complexity matching extends across individuals and their respective layers of processes when cooperating in joint tasks. We measured coordination in a joint tower building task through the layers of sound and movement patterns produced by partners and found that partners built higher towers when their sound patterns fell into more similar relations with each other across timescales, as measured by complexity matching. Our findings shed light on the function of complexity matching and lead to new hypotheses about multiscale coordination and communication. We discuss how complexity matching encompasses flexible and complementary dynamics between partners that support complex acts of human coordination.