Resource Management and Model Personalization for Federated Learning over Wireless Edge Networks

Client and Internet of Things devices are increasingly equipped with the ability to sense, process, and communicate data with high efficiency. This is resulting in a major shift in machine learning (ML) computation at the network edge. Distributed learning approaches such as federated learning that move ML training to end devices have emerged, promising lower latency and bandwidth costs and enhanced privacy of end users’ data. However, new challenges that arise from the heterogeneous nature of the devices’ communication rates, compute capabilities, and the limited observability of the training data at each device must be addressed. All these factors can significantly affect the training performance in terms of overall accuracy, model fairness, and convergence time. We present compute-communication and data importance-aware resource management schemes optimizing these metrics and evaluate the training performance on benchmark datasets. We also develop a federated meta-learning solution, based on task similarity, that serves as a sample efficient initialization for federated learning, as well as improves model personalization and generalization across non-IID (independent, identically distributed) data. We present experimental results on benchmark federated learning datasets to highlight the performance gains of the proposed methods in comparison to the well-known federated averaging algorithm and its variants.

Musculoskeletal Model Personalization Affects Metabolic Cost Estimates for Walking

Assessment of metabolic cost as a metric for human performance has expanded across various fields within the scientific, clinical, and engineering communities. As an alternative to measuring metabolic cost experimentally, musculoskeletal models incorporating metabolic cost models have been developed. However, to utilize these models for practical applications, the accuracy of their metabolic cost predictions requires improvement. Previous studies have reported the benefits of using personalized musculoskeletal models for various applications, yet no study has evaluated how model personalization affects metabolic cost estimation. This study investigated the effect of musculoskeletal model personalization on estimates of metabolic cost of transport (CoT) during post-stroke walking using three commonly used metabolic cost models. We analyzed walking data previously collected from two male stroke survivors with right-sided hemiparesis. The three metabolic cost models were implemented within three musculoskeletal modeling approaches involving different levels of personalization. The first approach used a scaled generic OpenSim model and found muscle activations via static optimization (SOGen). The second approach used a personalized electromyographic (EMG)-driven musculoskeletal model with personalized functional axes but found muscle activations via static optimization (SOCal). The third approach used the same personalized EMG-driven model but calculated muscle activations directly from EMG data (EMGCal). For each approach, the muscle activation estimates were used to calculate each subject’s CoT at different gait speeds using three metabolic cost models (Umberger et al., 2003; Bhargava et al., 2004; Umberger, 2010). The calculated CoT values were compared with published CoT data as a function of walking speed, step length asymmetry, stance time asymmetry, double support time asymmetry, and severity of motor impairment (i.e., Fugl-Meyer score). Overall, only SOCal and EMGCal with the Bhargava metabolic cost model were able to reproduce accurately published experimental trends between CoT and various clinical measures of walking asymmetry post-stroke. Tuning of the parameters in the different metabolic cost models could potentially resolve the observed CoT magnitude differences between model predictions and experimental measurements. Realistic CoT predictions may allow researchers to predict human performance, surgical outcomes, and rehabilitation outcomes reliably using computational simulations.

Replay Simulations with Personalized Metabolic Model for Treatment Design and Evaluation in Type 1 Diabetes

Background: The capacity to replay data collected in real life by people with type 1 diabetes mellitus (T1DM) would lead to individualized (vs population) assessment of treatment strategies to control blood glucose and possibly true personalization. Patek et al introduced such a technique, relying on regularized deconvolution of a population glucose homeostasis model to estimate a residual additive signal and reproduce the experimental data; therefore, allowing the subject-specific replay of what-if scenarios by altering the model inputs (eg, insulin). This early method was shown to have a limited domain of validity. We propose and test in silico a similar approach and extend the method applicability. Methods: A subject-specific model personalization of insulin sensitivity and meal-absorption parameters is performed. The University of Virginia (UVa)/Padova T1DM simulator is used to generate experimental scenarios and test the ability of the methodology to accurately reproduce changes in glucose concentration to alteration in meal and insulin inputs. Method performance is assessed by comparing true (UVa/Padova simulator) and replayed glucose traces, using the mean absolute relative difference (MARD) and the Clarke error grid analysis (CEGA). Results: Model personalization led to a 9.08 and 6.07 decrease in MARD over a prior published method of replaying altered insulin scenarios for basal and bolus changes, respectively. Replay simulations achieved high accuracy, with MARD <10% and more than 95% of readings falling in the CEGA A-B zones for a wide range of interventions. Conclusions: In silico studies demonstrate that the proposed method for replay simulation is numerically and clinically valid over broad changes in scenario inputs, indicating possible use in treatment optimization.

Distributed Training of ML Model on Mobile Devices

Currently, the need for distributed ML training solutions in the world is increasing. However, existing tools, in particular TensorFlow Federated, are at the very beginning of their development, difficult to implement, and currently suitable exclusively for simulation on servers. For mobile devices, reliable approaches for this purpose do not exist. This article has designed and presented an approach to such distributed training of the ML-model on mobile devices, implemented on existing technologies. It is based on the concept of model personalization. In this approach, this concept is improved as a consequence of mitigating the identified drawbacks. The implementation process is structured so that at all stages of working with the ML-model use only one Swift programming language (Swift for TensorFlow and Core ML 3 are used), making this approach even more convenient and reliable due to the common code base.

Musculoskeletal Model Personalization Affects Metabolic Cost Estimates for Walking

Assessment of metabolic energy cost as a metric for human performance has expanded across various fields within the scientific, clinical, and engineering communities. As an alternative to measuring metabolic cost experimentally, musculoskeletal models incorporating metabolic cost models have been developed. However, to utilize these models for practical applications, the accuracy of their metabolic cost predictions requires improvement. Previous studies have reported the benefits of using personalized musculoskeletal models for various applications, yet no study has evaluated how model personalization affects metabolic cost estimation. This study investigated the effect of musculoskeletal model personalization on estimates of metabolic cost of transport (CoT) during post-stroke walking using three commonly used metabolic cost models. We analyzed data previously collected from two male stroke survivors with right-sided hemiparesis. The three metabolic cost models were implemented within three musculoskeletal modeling approaches involving different levels of personalization. The first approach used a scaled generic OpenSim model and found muscle activations via static optimization (SOGen). The second approach used a personalized EMG-driven musculoskeletal model with personalized functional axes but found muscle activations via static optimization (SOCal). The third approach used the same personalized EMG-driven model but calculated muscle activations directly from EMG data (EMGCal). For each approach, the muscle activation estimates were used to calculate each subject’s cost of transport (CoT) at different gait speeds using three metabolic cost models (Umberger 2003, Umberger 2010, and Bhargava 2004). The calculated CoT values were compared with published CoT trends as a function of stance time, double support time, step positions, walking speed, and severity of motor impairment (i.e., Fugl-Meyer score). Overall, U10-SOCal, U10-EMGCal, U03-SOCal, and U03-EMGCal were able to produce slopes between CoT and the different measures of walking asymmetry that were statistically similar to those found in the literature. Although model personalization seemed to improve CoT estimates, further tuning of parameters associated with the different metabolic cost models in future studies may allow for realistic CoT predictions. An improvement in CoT predictions may allow researchers to predict human performance, surgical, and rehabilitation outcomes reliably using computational simulations.

Deep Match to Rank Model for Personalized Click-Through Rate Prediction

Click-through rate (CTR) prediction is a core task in the field of recommender system and many other applications. For CTR prediction model, personalization is the key to improve the performance and enhance the user experience. Recently, several models are proposed to extract user interest from user behavior data which reflects user's personalized preference implicitly. However, existing works in the field of CTR prediction mainly focus on user representation and pay less attention on representing the relevance between user and item, which directly measures the intensity of user's preference on target item. Motivated by this, we propose a novel model named Deep Match to Rank (DMR) which combines the thought of collaborative filtering in matching methods for the ranking task in CTR prediction. In DMR, we design User-to-Item Network and Item-to-Item Network to represent the relevance in two forms. In User-to-Item Network, we represent the relevance between user and item by inner product of the corresponding representation in the embedding space. Meanwhile, an auxiliary match network is presented to supervise the training and push larger inner product to represent higher relevance. In Item-to-Item Network, we first calculate the item-to-item similarities between user interacted items and target item by attention mechanism, and then sum up the similarities to obtain another form of user-to-item relevance. We conduct extensive experiments on both public and industrial datasets to validate the effectiveness of our model, which outperforms the state-of-art models significantly.