Combining haptics and inertial motion capture to enhance remote control of a dual-arm robot

High dexterity is required in tasks in which there is contact between objects, such as surface conditioning (wiping, polishing, scuffing, sanding, etc.), specially when the location of the objects involved is unknown or highly inaccurate because they are moving, like a car body in automotive industry lines. These applications require the human adaptability and the robot accuracy. However, sharing the same workspace is not possible in most cases due to safety issues. Hence, a multi-modal teleoperation system combining haptics and an inertial motion capture system is introduced in this work. The human operator gets the sense of touch thanks to haptic feedback, whereas using the motion capture device allows more naturalistic movements. Visual feedback assistance is also introduced to enhance immersion. A Baxter dual-arm robot is used to offer more flexibility and manoeuvrability, allowing to perform two independent operations simultaneously. Several tests have been carried out to assess the proposed system. As it is shown by the experimental results, the task duration is reduced and the overall performance improves thanks to the proposed teleoperation method.

Informing the design of a multisensory learning environment for elementary mathematics learning

It is well known that primary school children may face difficulties in acquiring mathematical competence, possibly because teaching is generally based on formal lessons with little opportunity to exploit more multisensory-based activities within the classroom. To overcome such difficulties, we report here the exemplary design of a novel multisensory learning environment for teaching mathematical concepts based on meaningful inputs from elementary school teachers. First, we developed and administered a questionnaire to 101 teachers asking them to rate based on their experience the learning difficulty for specific arithmetical and geometrical concepts encountered by elementary school children. Additionally, the questionnaire investigated the feasibility to use multisensory information to teach mathematical concepts. Results show that challenging concepts differ depending on children school level, thus providing a guidance to improve teaching strategies and the design of new and emerging learning technologies accordingly. Second, we obtained specific and practical design inputs with workshops involving elementary school teachers and children. Altogether, these findings are used to inform the design of emerging multimodal technological applications, that take advantage not only of vision but also of other sensory modalities. In the present work, we describe in detail one exemplary multisensory environment design based on the questionnaire results and design ideas from the workshops: the Space Shapes game, which exploits visual and haptic/proprioceptive sensory information to support mental rotation, 2D–3D transformation and percentages. Corroborating research evidence in neuroscience and pedagogy, our work presents a functional approach to develop novel multimodal user interfaces to improve education in the classroom.