Analysis of postures for handwriting on touch screens without using tools

The act of handwriting affected the evolutionary development of humans and still impacts the motor cognition of individuals. However, the ubiquitous use of digital technologies has drastically decreased the number of times we really need to pick a pen up and write on paper. Nonetheless, the positive cognitive impact of handwriting is widely recognized, and a possible way to merge the benefits of handwriting and digital writing is to use suitable tools to write over touchscreens or graphics tablets. In this manuscript, we focus on the possibility of using the hand itself as a writing tool. A novel hand posture named FingerPen is introduced, and can be seen as a grasp performed by the hand on the index finger. A comparison with the most common posture that people tend to assume (i.e. index finger-only exploitation) is carried out by means of a biomechanical model. A conducted user study shows that the FingerPen is appreciated by users and leads to accurate writing traits.

Hand Pose Recognition Using Parallel Multi Stream CNN

Recently, several computer applications provided operating mode through pointing fingers, waving hands, and with body movement instead of a mouse, keyboard, audio, or touch input such as sign language recognition, robot control, games, appliances control, and smart surveillance. With the increase of hand-pose-based applications, new challenges in this domain have also emerged. Support vector machines and neural networks have been extensively used in this domain using conventional RGB data, which are not very effective for adequate performance. Recently, depth data have become popular due to better understating of posture attributes. In this study, a multiple parallel stream 2D CNN (two-dimensional convolution neural network) model is proposed to recognize the hand postures. The proposed model comprises multiple steps and layers to detect hand poses from image maps obtained from depth data. The hyper parameters of the proposed model are tuned through experimental analysis. Three publicly available benchmark datasets: Kaggle, First Person, and Dexter, are used independently to train and test the proposed approach. The accuracy of the proposed method is 99.99%, 99.48%, and 98% using the Kaggle hand posture dataset, First Person hand posture dataset, and Dexter dataset, respectively. Further, the results obtained for F1 and AUC scores are also near-optimal. Comparative analysis with state-of-the-art shows that the proposed model outperforms the previous methods.

sEMG-Based Hand Posture Recognition Considering Electrode Shift, Feature Vectors, and Posture Groups

Surface electromyography (sEMG)-based gesture recognition systems provide the intuitive and accurate recognition of various gestures in human-computer interaction. In this study, an sEMG-based hand posture recognition algorithm was developed, considering three main problems: electrode shift, feature vectors, and posture groups. The sEMG signal was measured using an armband sensor with the electrode shift. An artificial neural network classifier was trained using 21 feature vectors for seven different posture groups. The inter-session and inter-feature Pearson correlation coefficients (PCCs) were calculated. The results indicate that the classification performance improved with the number of training sessions of the electrode shift. The number of sessions necessary for efficient training was four, and the feature vectors with a high inter-session PCC (r > 0.7) exhibited high classification accuracy. Similarities between postures in a posture group decreased the classification accuracy. Our results indicate that the classification accuracy could be improved with the addition of more electrode shift training sessions and that the PCC is useful for selecting the feature vector. Furthermore, hand posture selection was as important as feature vector selection. These findings will help in optimizing the sEMG-based pattern recognition algorithm more easily and quickly.

The effect of hand posture on swimming efficiency

Our quest is for the thumb and finger positions that maximize drag in front crawl swimming and thus maximize propulsion efficiency. We focus on drag in a stationary flow. Swimming is in water, but using Reynolds similarity the drag experiments are done in a wind tunnel. We measure the forces on real-life models of a forearm with hands, flexing the thumb and fingers in various positions. We study the influence on drag of cupping the hand and flexing the thumb. We find that cupping the hand is detrimental for drag. Swimming is most efficient with a flat hand. Flexing the thumb has a small effect on the drag, such that the drag is largest for the opened (abducted) thumb. Flow structures around the hand are visualized using robotic volumetric particle image velocimetry. From the time-averaged velocity fields we reconstruct the pressure distribution on the hand. These pressures are compared to the result of a direct measurement. The reached accuracy of $$\approx$$ ≈ 10% does not yet suffice to reproduce the small drag differences between the hand postures. Graphical Abstract

Typealike

We propose a style of hand postures to trigger commands on a laptop. The key idea is to perform hand-postures while keeping the hands on, beside, or below the keyboard, to align with natural laptop usage. 36 hand-posture variations are explored considering three resting locations, left or right hand, open or closed hand, and three wrist rotation angles. A 30-participant formative study measures posture preferences and generates a dataset of nearly 350K images under different lighting conditions and backgrounds. A deep learning recognizer achieves over 97% accuracy when classifying all 36 postures with 2 additional non-posture classes for typing and non-typing. A second experiment with 20 participants validates the recognizer under real-time usage and compares posture invocation time with keyboard shortcuts. Results find low error rates and fast formation time, indicating postures are close to current typing and pointing postures. Finally, practical use case demonstrations are presented, and further extensions discussed.

Framework of Mobile Game Design as an Assistive Technology for Children with Motor Disabilities

Computer games have emerged in the past decade as potential media beyond entertainment. Despite its popularity, game accessibility remains a major concern of various researchers. Children population with motor disabilities is a potential target for developing entertainment or therapeutic support games due to their interest to play. This paper presents: (1) a framework for mobile games for children with motor disability using simple hand postures and (2) Xgboost decision tree as a hand posture recognizer (98.48 percent training accuracy and 96.76 percent testing accuracy) as a prototype of hand posture-based commands as assistive technology to interact with games.