Online convex combination of ranking models

As a task of high importance for recommender systems, we consider the problem of learning the convex combination of ranking algorithms by online machine learning. First, we propose a stochastic optimization algorithm that uses finite differences. Our new algorithm achieves close to optimal empirical performance for two base rankers, while scaling well with an increased number of models. In our experiments with five real-world recommendation data sets, we show that the combination offers significant improvement over previously known stochastic optimization techniques. The proposed algorithm is the first effective stochastic optimization method for combining ranked recommendation lists by online machine learning. Secondly, we propose an exponentially weighted algorithm based on a grid over the space of combination weights. We show that the algorithm has near-optimal worst-case performance bound. The bound provides the first theoretical guarantee for non-convex bandits using limited number of evaluations under very general conditions.

Guaranteed cost controller for discrete time-delayed systems with actuator saturation

The problem of guaranteed cost (GC) control using static-state feedback controllers for uncertain linear discrete time-delayed systems subjected to actuator saturation is studied in this paper. The stability analysis of closed-loop systems is carried out using a Lyapunov-Krasovskii functional. Conditions for the existence of state-feedback GC controllers are developed using a linear matrix inequality (LMI)-based criterion. The approach ensures a sufficient performance bound over all the acceptable parameter uncertainties. The scheme of the optimal GC controller problem is framed as a convex optimization problem with LMI constraints. The design of GC controllers for discrete-time systems subjected to actuator saturation without considering the effect of state-delay is also discussed. The effectiveness of the proposed approach is illustrated using suitable examples.

The Wisdom of Model Crowds

A wide body of empirical research has revealed the descriptive shortcomings of expected value and expected utility models of risky decision making. In response, numerous models have been advanced to predict and explain people’s choices between gambles. Although some of these models have had a great impact in the behavioral, social, and management sciences, there is little consensus about which model offers the best account of choice behavior. In this paper, we conduct a large-scale comparison of 58 prominent models of risky choice, using 19 existing behavioral data sets involving more than 800 participants. This allows us to comprehensively evaluate models in terms of individual-level predictive performance across a range of different choice settings. We also identify the psychological mechanisms that lead to superior predictive performance and the properties of choice stimuli that favor certain types of models over others. Moreover, drawing on research on the wisdom of crowds, we argue that each of the existing models can be seen as an expert that provides unique forecasts in choice predictions. Consistent with this claim, we find that crowds of risky choice models perform better than individual models and thus provide a performance bound for assessing the historical accumulation of knowledge in our field. Our results suggest that each model captures unique aspects of the decision process and that existing risky choice models offer complementary rather than competing accounts of behavior. We discuss the implications of our results on theories of risky decision making and the quantitative modeling of choice behavior. This paper was accepted by Yuval Rottenstreich, behavioral economics and decision analysis.

Theoretical Performance Bounds of Model-Based Electrocardiogram Parameter Estimation

<div>Objective: Clinical parameter estimation from the electrocardiogram (ECG) is a recurrent field of research. It is debated that ECG parameter estimation performed by human experts and machines/algorithms is always model-based (implicitly or explicitly). Therefore, depending on the selected data-model, the adopted estimation scheme (least-squares error, maximum likelihood, or Bayesian), and the prior assumptions on the model parameters and noise distributions, any estimation algorithm used in this context has an upper performance bound, which is not exceedable (for the same model and assumptions).</div><div><br></div><div>Method: In this research, we develop a comprehensive theoretical framework for ECG parameter estimation and derive the Cramér-Rao lower bounds (CRLBs) for the most popular signal models used in the ECG modeling literature; namely bases expansions (including polynomials) and sum of Gaussian functions.</div><div><br></div><div>Results: The developed framework is evaluated over real and synthetic data, for three popular applications: T/R ratio estimation, ST-segment analysis and QT-interval estimation, using the state-of-the-art estimators in each context, and compared with the derived theoretical CRLBs.</div><div>Conclusion and Significance: The proposed framework and the derived CRLBs provide fact-based guidelines for the selection of data-models, sampling frequency (beyond the Nyquist requirements), modeling segment length, the number of beats required for average ECG beat extraction, and other factors that influence the accuracy of ECG-based clinical parameter estimation.</div>

Theoretical Performance Bounds of Model-Based Electrocardiogram Parameter Estimation

<div>Objective: Clinical parameter estimation from the electrocardiogram (ECG) is a recurrent field of research. It is debated that ECG parameter estimation performed by human experts and machines/algorithms is always model-based (implicitly or explicitly). Therefore, depending on the selected data-model, the adopted estimation scheme (least-squares error, maximum likelihood, or Bayesian), and the prior assumptions on the model parameters and noise distributions, any estimation algorithm used in this context has an upper performance bound, which is not exceedable (for the same model and assumptions).</div><div><br></div><div>Method: In this research, we develop a comprehensive theoretical framework for ECG parameter estimation and derive the Cramér-Rao lower bounds (CRLBs) for the most popular signal models used in the ECG modeling literature; namely bases expansions (including polynomials) and sum of Gaussian functions.</div><div><br></div><div>Results: The developed framework is evaluated over real and synthetic data, for three popular applications: T/R ratio estimation, ST-segment analysis and QT-interval estimation, using the state-of-the-art estimators in each context, and compared with the derived theoretical CRLBs.</div><div>Conclusion and Significance: The proposed framework and the derived CRLBs provide fact-based guidelines for the selection of data-models, sampling frequency (beyond the Nyquist requirements), modeling segment length, the number of beats required for average ECG beat extraction, and other factors that influence the accuracy of ECG-based clinical parameter estimation.</div>

Theoretical Performance Bounds of Model-Based Electrocardiogram Parameter Estimation

<div>Objective: Clinical parameter estimation from the electrocardiogram (ECG) is a recurrent field of research. It is debated that ECG parameter estimation performed by human experts and machines/algorithms is always model-based (implicitly or explicitly). Therefore, depending on the selected data-model, the adopted estimation scheme (least-squares error, maximum likelihood, or Bayesian), and the prior assumptions on the model parameters and noise distributions, any estimation algorithm used in this context has an upper performance bound, which is not exceedable (for the same model and assumptions).</div><div><br></div><div>Method: In this research, we develop a comprehensive theoretical framework for ECG parameter estimation and derive the Cramér-Rao lower bounds (CRLBs) for the most popular signal models used in the ECG modeling literature; namely bases expansions (including polynomials) and sum of Gaussian functions.</div><div><br></div><div>Results: The developed framework is evaluated over real and synthetic data, for three popular applications: T/R ratio estimation, ST-segment analysis and QT-interval estimation, using the state-of-the-art estimators in each context, and compared with the derived theoretical CRLBs.</div><div>Conclusion and Significance: The proposed framework and the derived CRLBs provide fact-based guidelines for the selection of data-models, sampling frequency (beyond the Nyquist requirements), modeling segment length, the number of beats required for average ECG beat extraction, and other factors that influence the accuracy of ECG-based clinical parameter estimation.</div>

Average-Reward Reinforcement Learning with Trust Region Methods

Most of reinforcement learning algorithms optimize the discounted criterion which is beneficial to accelerate the convergence and reduce the variance of estimates. Although the discounted criterion is appropriate for certain tasks such as financial related problems, many engineering problems treat future rewards equally and prefer a long-run average criterion. In this paper, we study the reinforcement learning problem with the long-run average criterion. Firstly, we develop a unified trust region theory with discounted and average criteria. With the average criterion, a novel performance bound within the trust region is derived with the Perturbation Analysis (PA) theory. Secondly, we propose a practical algorithm named Average Policy Optimization (APO), which improves the value estimation with a novel technique named Average Value Constraint. To the best of our knowledge, our work is the first one to study the trust region approach with the average criterion and it complements the framework of reinforcement learning beyond the discounted criterion. Finally, experiments are conducted in the continuous control environment MuJoCo. In most tasks, APO performs better than the discounted PPO, which demonstrates the effectiveness of our approach.

Low-Latency Short-Packet Transmission over a Large Spatial Scale

Short-packet transmission has attracted considerable attention due to its potential to achieve ultralow latency in automated driving, telesurgery, the Industrial Internet of Things (IIoT), and other applications emerging in the coming era of the Six-Generation (6G) wireless networks. In 6G systems, a paradigm-shifting infrastructure is anticipated to provide seamless coverage by integrating low-Earth orbit (LEO) satellite networks, which enable long-distance wireless relaying. However, how to efficiently transmit short packets over a sizeable spatial scale remains open. In this paper, we are interested in low-latency short-packet transmissions between two distant nodes, in which neither propagation delay, nor propagation loss can be ignored. Decode-and-forward (DF) relays can be deployed to regenerate packets reliably during their delivery over a long distance, thereby reducing the signal-to-noise ratio (SNR) loss. However, they also cause decoding delay in each hop, the sum of which may become large and cannot be ignored given the stringent latency constraints. This paper presents an optimal relay deployment to minimize the error probability while meeting both the latency and transmission power constraints. Based on an asymptotic analysis, a theoretical performance bound for distant short-packet transmission is also characterized by the optimal distance–latency–reliability tradeoff, which is expected to provide insights into designing integrated LEO satellite communications in 6G.

Corrigendum: Greed Works—Online Algorithms for Unrelated Machine Stochastic Scheduling

This corrigendum fixes an incorrect claim in the paper Gupta et al. [Gupta V, Moseley B, Uetz M, Xie Q (2020) Greed works—online algorithms for unrelated machine stochastic scheduling. Math. Oper. Res. 45(2):497–516.], which led us to claim a performance guarantee of 6 for a greedy algorithm for deterministic online scheduling with release times on unrelated machines. The result is based on an upper bound on the increase of the objective function value when adding an additional job [Formula: see text] to a machine [Formula: see text] (Gupta et al., lemma 6). It was pointed out by Sven Jäger from Technische Universität Berlin that this upper bound may fail to hold. We here present a modified greedy algorithm and analysis, which leads to a performance guarantee of 7.216 instead. Correspondingly, also the claimed performance guarantee of [Formula: see text] in theorem 4 of Gupta et al. for the stochastic online problem has to be corrected. We obtain a performance bound [Formula: see text].