A Smartphone-Based Cursor Position System in Cross-Device Interaction Using Machine Learning Techniques

The use of mobile devices, especially smartphones, has become popular in recent years. There is an increasing need for cross-device interaction techniques that seamlessly integrate mobile devices and large display devices together. This paper develops a novel cross-device cursor position system that maps a mobile device’s movement on a flat surface to a cursor’s movement on a large display. The system allows a user to directly manipulate objects on a large display device through a mobile device and supports seamless cross-device data sharing without physical distance restrictions. To achieve this, we utilize sound localization to initialize the mobile device position as the starting location of a cursor on the large screen. Then, the mobile device’s movement is detected through an accelerometer and is accordingly translated to the cursor’s movement on the large display using machine learning models. In total, 63 features and 10 classifiers were employed to construct the machine learning models for movement detection. The evaluation results have demonstrated that three classifiers, in particular, gradient boosting, linear discriminant analysis (LDA), and naïve Bayes, are suitable for detecting the movement of a mobile device.

Comparison of Pixel-based Position Input and Direct Acceleration Input for Virtual Stick Balancing Tests

A virtual stick balancing environment is developed using a computer mouse as input device. The development process is presented both on the hardware and software level. Two possible concepts are suggested to obtain the acceleration of the input device: discrete differentiation of the cursor position measured in pixels on the screen and by direct measurements via an Inertial Measurement Unit (IMU). The comparison of the inputs is carried out with test measurements using a crank mechanism. The measured signals are compared to the prescribed motion of the mechanism and it is shown that the IMU-based input signal fits better to the prescribed motion than the pixel-based input signal. The pixel-based input can also be applied after additional filtering, but this presents an extra computational delay in the feedback loop.

Chronic stroke survivors show task-dependent modulation of motor variability during bimanual coordination

Stroke often results in hemiparesis, leaving one side of the body “affected” relative to the other side. Prior research has shown that the affected arm has higher variability; however, the extent to which this variability can be modulated is unclear. Here we used a shared bimanual task to examine the degree to which participants could modulate the variability in the affected arm after stroke. Participants with chronic stroke ( n = 11) and age-matched controls ( n = 11) performed unimanual and bimanual reaching movements to move a cursor on a screen to different targets. In the unimanual condition, the cursor was controlled only by the movement of a single arm, whereas, in the bimanual condition, the cursor position was “shared” between the two arms by using a weighted average of the two hand positions. Unknown to the participants, we altered the weightings of the affected and unaffected arms to cursor motion and examined how the movement variability on each arm changed depending on its contribution to the task. Results showed that stroke survivors had higher movement variability on the affected arm; however, like age-matched controls, they were able to modulate the variability in both the affected and unaffected arms according to the weighting condition. Specifically, as the weighting on a particular arm increased (i.e., it became more important to the task), the movement variability decreased. These results show that stroke survivors are capable of modulating variability depending on the task context, and this feature may potentially be exploited for rehabilitation paradigms. NEW & NOTEWORTHY We show that chronic stroke survivors, similar to age-matched controls, are able to modulate variability in their affected and unaffected limbs in redundant bimanual tasks as a function of how these limbs contribute to the task. Specifically, in both affected and unaffected limbs, the variability of the limb increases as its contribution to the task decreases. This feature may potentially be exploited in rehabilitation paradigms using bimanual tasks.

Stroke survivors show task-dependent modulation of motor variability during bimanual coordination

ABSTRACTStroke often results in hemiparesis, leaving one side of the body ‘affected’ relative to the other side. Prior research has shown that the affected arm has higher variability – however, the extent to which this variability can be modulated is unclear. Here we used a shared bimanual task to examine the degree to which participants could modulate the variability in the affected arm after stroke. Participants with chronic stroke (n = 11), and age-matched controls (n = 11) performed unimanual and bimanual reaching movements to move a cursor on a screen to different targets. In the unimanual condition, the cursor was controlled only by the movement of a single arm whereas in the bimanual condition, the cursor position was “shared” between the two arms by using a weighted average of the two hand positions. Unknown to the participants, we altered the weightings of the affected and unaffected arms to cursor motion and examined how the movement variability on each arm changed depending on its contribution to the task. Results showed that stroke survivors had higher movement variability on the affected arm – however, like age-matched controls, they were able to modulate the variability in both the affected and unaffected arms according to the weighting condition. Specifically, as the weighting on a particular arm increased (i.e. it became more important to the task), the movement variability decreased. These results show that stroke survivors are capable of modulating variability depending on the task context, and this feature may potentially be exploited for rehabilitation paradigms

Cross talk in implicit assignment of error information during bimanual visuomotor learning

When a neural movement controller, called an “internal model,” is adapted to a novel environment, the movement error needs to be appropriately associated with the controller. However, their association is not necessarily guaranteed for bimanual movements in which two controllers—one for each hand—result in two movement errors. Considering the implicit nature of the adaptation process, the movement error of one hand can be erroneously associated with the controller of the other hand. Here, we investigated this credit-assignment problem in bimanual movement by having participants perform bimanual, symmetric back-and-forth movements while displaying the position of the right hand only with a cursor. In the training session, the cursor position was gradually rotated clockwise, such that the participants were unaware of the rotation. The movement of the right hand gradually rotated counterclockwise as a consequence of adaptation. Although the participants knew that the cursor reflected the movement of the right hand, such gradual adaptation was also observed for the invisible left hand, especially when the cursor was presented on the left side of the display. Thus the movement error of the right hand was implicitly assigned to the left-hand controller. Such cross talk in credit assignment might influence motor adaptation performance, even when two cursors are presented; the adaptation was impaired when the rotations imposed on the cursors were opposite compared with when they were in the same direction. These results indicate the inherent presence of cross talk in the process of associating action with consequence in bimanual movement.