Creating and Augmenting Keyboards for Extended Reality with the K eyboard A ugmentation T oolkit

This article discusses the Keyboard Augmentation Toolkit (KAT), which supports the creation of virtual keyboards that can be used both for standalone input (e.g., for mid-air text entry) and to augment physically tracked keyboards/surfaces in mixed reality. In a user study, we firstly examine the impact and pitfalls of visualising shortcuts on a tracked physical keyboard, exploring the utility of virtual per-keycap displays. Supported by this and other recent developments in XR keyboard research, we then describe the design, development, and evaluation-by-demonstration of KAT. KAT simplifies the creation of virtual keyboards (optionally bound to a tracked physical keyboard) that support enhanced display —2D/3D per-key content that conforms to the virtual key bounds; enhanced interactivity —supporting extensible per-key states such as tap, dwell, touch, swipe; flexible keyboard mappings that can encapsulate groups of interaction and display elements, e.g., enabling application-dependent interactions; and flexible layouts —allowing the virtual keyboard to merge with and augment a physical keyboard, or switch to an alternate layout (e.g., mid-air) based on need. Through these features, KAT will assist researchers in the prototyping, creation and replication of XR keyboard experiences, fundamentally altering the keyboard’s form and function.

On the divisibility among power GCD and power LCM matrices on gcd-closed sets

Let [Formula: see text] and [Formula: see text] be positive integers and let [Formula: see text] be a set of [Formula: see text] distinct positive integers. For [Formula: see text], one defines [Formula: see text]. We denote by [Formula: see text] (respectively, [Formula: see text]) the [Formula: see text] matrix having the [Formula: see text]th power of the greatest common divisor (respectively, the least common multiple) of [Formula: see text] and [Formula: see text] as its [Formula: see text]-entry. In this paper, we show that for arbitrary positive integers [Formula: see text] and [Formula: see text] with [Formula: see text], the [Formula: see text]th power matrices [Formula: see text] and [Formula: see text] are both divisible by the [Formula: see text]th power matrix [Formula: see text] if [Formula: see text] is a gcd-closed set (i.e. [Formula: see text] for all integers [Formula: see text] and [Formula: see text] with [Formula: see text]) such that [Formula: see text]. This confirms two conjectures of Shaofang Hong proposed in 2008.

On the inverse sum indeg energy of trees

Let [Formula: see text] be a graph of order [Formula: see text] and [Formula: see text] be the degree of the vertex [Formula: see text], for [Formula: see text]. The [Formula: see text] matrix of [Formula: see text] is the square matrix of order [Formula: see text] whose [Formula: see text]-entry is equal to [Formula: see text] if [Formula: see text] is adjacent to [Formula: see text], and zero otherwise. Let [Formula: see text], be the eigenvalues of [Formula: see text] matrix. The [Formula: see text] energy of a graph [Formula: see text], denoted by [Formula: see text], is defined as the sum of the absolute values of the eigenvalues of [Formula: see text] matrix. In this paper, we prove that the star has the minimum [Formula: see text] energy among trees.

Write, Attend and Spell

Text entry on a smartwatch is challenging due to its small form factor. Handwriting recognition using the built-in sensors of the watch (motion sensors, microphones, etc.) provides an efficient and natural solution to deal with this issue. However, prior works mainly focus on individual letter recognition rather than word recognition. Therefore, they need users to pause between adjacent letters for segmentation, which is counter-intuitive and significantly decreases the input speed. In this paper, we present 'Write, Attend and Spell' (WriteAS), a word-level text-entry system which enables free-style handwriting recognition using the motion signals of the smartwatch. First, we design a multimodal convolutional neural network (CNN) to abstract motion features across modalities. After that, a stacked dilated convolutional network with an encoder-decoder network is applied to get around letter segmentation and output words in an end-to-end way. More importantly, we leverage a multi-task sequence learning method to enable handwriting recognition in a streaming way. We construct the first sequence-to-sequence handwriting dataset using smartwatch. WriteAS can yield 9.3% character error rate (CER) on 250 words for new users and 3.8% CER for words unseen in the training set. In addition, WriteAS can handle various writing conditions very well. Given the promising performance, we envision that WriteAS can be a fast and accurate input tool for smartwatch.

Type Anywhere You Want: An Introduction to Invisible Mobile Keyboard

Contemporary soft keyboards possess limitations: the lack of physical feedback results in an increase of typos, and the interface of soft keyboards degrades the utility of the screen. To overcome these limitations, we propose an Invisible Mobile Keyboard (IMK), which lets users freely type on the desired area without any constraints. To facilitate a data-driven IMK decoding task, we have collected the most extensive text-entry dataset (approximately 2M pairs of typing positions and the corresponding characters). Additionally, we propose our baseline decoder along with a semantic typo correction mechanism based on self-attention, which decodes such unconstrained inputs with high accuracy (96.0%). Moreover, the user study reveals that the users could type faster and feel convenience and satisfaction to IMK with our decoder. Lastly, we make the source code and the dataset public to contribute to the research community.