On the Unification of Common Actigraphic Data Scoring Algorithms

Actigraphy is a well-known, inexpensive method to investigate human movement patterns. Sleep and circadian rhythm studies are among the most popular applications of actigraphy. In this study, we investigate seven common sleep-wake scoring algorithms designed for actigraphic data, namely Cole-Kripke algorithm, two versions of Sadeh algorithm, Sazonov algorithm, Webster algorithm, UCSD algorithm and Scripps Clinic algorithm. We propose a unified mathematical framework describing five of them. One of the observed novelties is that five of these algorithms are in fact equivalent to low-pass FIR filters with very similar characteristics. We also provide explanations about the role of some factors defining these algorithms, as none were given by their Authors who followed empirical procedures. Proposed framework provides a robust mathematical description of discussed algorithms, which for the first time allows one to fully understand their operation and basics.

PP17 Development and application of a clinical risk stratification tool using routine data from electronic patient clinical records

BackgroundFollowing the introduction of electronic patient clinical records, ambulance service managers wished to combine clinical and operational data to devise a method of risk stratifying 999 calls by the MPDS disposition code assigned at call triage. Initial aims were to establish the risk threshold if an ambulance was no longer routinely dispatched.MethodsData selected were representative of high or low clinical risk, and reliably recorded. The following ‘risk factors’ were chosen:Call outcomeEmergency conditionsClinical interventionsMedications administeredWith expert local opinion, a scoring algorithm was created using weighted factor scores to create an aggregate risk score for each MPDS code. It was also designed to distribute codes along a ‘risk range’, allowing for thresholds setting suitable to the specific purpose of individual projects. These factors and their scores were captured alongside contextual information and to date contains over 1.4 million records over 3 years.In collaboration with academic colleagues, we also developed an AI model to refine the algorithm used to reflect acuity. With one year of data the tool did not demonstrate the sensitivity or specificity to reliably contribute to prediction, however this exercise may be repeated now there is a greater volume of data.Applications: This Tool has been successfully used for a variety of purposes:Developing the Enhanced Hear and Treat policyAssessing risk of code downgrades in the pandemic responseIdentifying codes suitable for automatic specialist clinician allocationsSupplementing analysis of harm caused by long response delaysIdentifying codes for protection within End of Shift protocolsProviding intelligence to aid national decisions on code categorisationNext steps: The Tool continues to assist in decision-making locally. Future ambitions include:Validation of the scoring algorithmsProcess automation to ensure more timely data is availableCollaboration to improve the variety and volume of data

Evaluation of Automated Hypnogram Analysis on Multi-Scored Polysomnographies

A new method for automated sleep stage scoring of polysomnographies is proposed that uses a random forest approach to model feature interactions and temporal effects. The model mostly relies on features based on the rules from the American Academy of Sleep Medicine, which allows medical experts to gain insights into the model. A common way to evaluate automated approaches to constructing hypnograms is to compare the one produced by the algorithm to an expert's hypnogram. However, given the same data, two expert annotators will construct (slightly) different hypnograms due to differing interpretations of the data or individual mistakes. A thorough evaluation of our method is performed on a multi-labeled dataset in which both the inter-rater variability as well as the prediction uncertainties are taken into account, leading to a new standard for the evaluation of automated sleep stage scoring algorithms. On all epochs, our model achieves an accuracy of 82.7%, which is only slightly lower than the inter-rater disagreement. When only considering the 63.3% of the epochs where both the experts and algorithm are certain, the model achieves an accuracy of 97.8%. Transition periods between sleep stages are identified and studied for the first time. Scoring guidelines for medical experts are provided to complement the certain predictions by scoring only a few epochs manually. This makes the proposed method highly time-efficient while guaranteeing a highly accurate final hypnogram.

Integration of Spatial PD-L1 Expression with the Tumor Immune Microenvironment Outperforms Standard PD-L1 Scoring in Outcome Prediction of Urothelial Cancer Patients

Background: Immune therapy has gained significant importance in managing urothelial cancer. The value of PD-L1 remains a matter of controversy, thus requiring an in-depth analysis of its biological and clinical relevance. Methods: A total of 193 tumors of muscle-invasive bladder cancer patients (MIBC) were assessed with four PD-L1 assays. PD-L1 scoring results were correlated with data from a comprehensive digital-spatial immune-profiling panel using descriptive statistics, hierarchical clustering and uni-/multivariable survival analyses. Results: PD-L1 scoring algorithms are heterogeneous (agreements from 63.1% to 87.7%), and stems from different constellations of immune and tumor cells (IC/TC). While Ventana IC5% algorithm identifies tumors with high inflammation and favorable baseline prognosis, CPS10 and the TCarea25%/ICarea25% algorithm identify tumors with TC and IC expression. Spatially organized immune phenotypes, which correlate either with high PD-L1 IC expression and favorable prognosis or constitutive PD-L1 TC expression and poor baseline prognosis, cannot be resolved properly by PD-L1 algorithms. PD-L1 negative tumors with relevant immune infiltration can be detected by sTILs scoring on HE slides and digital CD8+ scoring. Conclusions: Contemporary PD-L1 scoring algorithms are not sufficient to resolve spatially distributed MIBC immune phenotypes and their clinical implications. A more comprehensive view of immune phenotypes along with the integration of spatial PD-L1 expression on IC and TC is necessary in order to stratify patients for ICI.

MUPPscore: An R script for expected a posteriori scoring of multidimensional pairwise preference items

In this manual, we present a flexible and freely available tool for obtaining latent trait scores from multi-unidimensional pairwise preference (MUPP) tests: An R script named MUPPscore. The development of the MUPPscore script provides a solution to the issue that is the previously inconvenient estimation of forced choice item pairs. Instead of using the computationally-intensive multidimensional Bayes modal procedure, the MUPPscore script employs the expected a posterior (EAP) scoring procedure, which provides plausible latent trait score estimates and is also consistent with scoring algorithms used in existing software programs intended for single stimulus measures (e.g., GGUM2004, IRTPRO). The MUPPscore script also returns the empirical marginal reliability of EAP theta estimates and outputs a series of files that can be used to easily create and modify three-dimensional surface charts for plotting MUPP item response function (IRF) in Microsoft Excel.

251 Agreement and reliability of a new polysomnography sleep staging algorithm against multiple human scorers

Introduction Scoring algorithms have the potential to increase polysomnography (PSG) scoring efficiency while also ensuring consistency and reproducibility. We sought to validate an updated sleep staging algorithm (Somnolyzer; Philips, Monroeville PA USA) against manual sleep staging, by analyzing a dataset we have previously used to report sleep staging variability across nine center-members of the Sleep Apnea Global Interdisciplinary Consortium (SAGIC). Methods Fifteen PSGs collected at a single sleep clinic were scored independently by technologists at nine SAGIC centers located in six countries, and auto-scored with the algorithm. Each 30-second epoch was staged manually according to American Academy of Sleep Medicine criteria. We calculated the intraclass correlation coefficient (ICC) and performed a Bland-Altman analysis comparing the average manual- and auto-scored total sleep time (TST) and time in each sleep stage (N1, N2, N3, rapid eye movement [REM]). We hypothesized that the values from auto-scoring would show good agreement and reliability when compared to the average across manual scorers. Results The participants contributing to the original dataset had a mean (SD) age of 47 (12) years and 80% were male. Auto-scoring showed substantial (ICC=0.60-0.80) or almost perfect (ICC=0.80-1.00) reliability compared to manual-scoring average, with ICCs (95% confidence interval) of 0.976 (0.931, 0.992) for TST, 0.681 (0.291, 0.879) for time in N1, 0.685 (0.299, 0.881) for time in N2, 0.922 (0.791, 0.973) for time in N3, and 0.930 (0.811, 0.976) for time in REM. Similarly, Bland-Altman analyses showed good agreement between methods, with a mean difference (limits of agreement) of only 1.2 (-19.7, 22.0) minutes for TST, 13.0 (-18.2, 44.1) minutes for N1, -13.8 (-65.7, 38.1) minutes for N2, -0.33 (-26.1, 25.5) minutes for N3, and -1.2 (-25.9, 23.5) minutes for REM. Conclusion Results support high reliability and good agreement between the auto-scoring algorithm and average human scoring for measurements of sleep durations. Auto-scoring slightly overestimated N1 and underestimated N2, but results for TST, N3 and REM were nearly identical on average. Thus, the auto-scoring algorithm is acceptable for sleep staging when compared against human scorers. Support (if any) Philips.

Glimpse: A Gaze-Based Measure of Temporal Salience

Temporal salience considers how visual attention varies over time. Although visual salience has been widely studied from a spatial perspective, its temporal dimension has been mostly ignored, despite arguably being of utmost importance to understand the temporal evolution of attention on dynamic contents. To address this gap, we proposed Glimpse, a novel measure to compute temporal salience based on the observer-spatio-temporal consistency of raw gaze data. The measure is conceptually simple, training free, and provides a semantically meaningful quantification of visual attention over time. As an extension, we explored scoring algorithms to estimate temporal salience from spatial salience maps predicted with existing computational models. However, these approaches generally fall short when compared with our proposed gaze-based measure. Glimpse could serve as the basis for several downstream tasks such as segmentation or summarization of videos. Glimpse’s software and data are publicly available.

Pairwise comparison based ranking and scoring algorithms

The task is to convert a large sparse matrix of unbalanced paired comparison vote counts to scores for each item.Five procedures are presented and benchmarked. It is recommended to use the BTL algorithm for full batch computations (e.g. one-time quantitative studies), and the proposed p-value based procedure when scores must be partially updated in production deployments.

Analysis of immune subtypes across the epithelial-mesenchymal plasticity spectrum

Epithelial-mesenchymal plasticity plays a critical role in many solid tumor types as a mediator of metastatic dissemination and treatment resistance. In addition, there is also a growing appreciation that the epithelial/mesenchymal status of a tumor plays a role in immune evasion and immune suppression. A deeper understanding of the immunological features of different tumor types has been facilitated by the availability of large gene expression datasets and the development of methods to deconvolute bulk RNA-Seq data. These resources have generated powerful new ways of characterizing tumors, including classification of immune subtypes based on differential expression of immunological genes. In the present work, we combine scoring algorithms to quantify epithelial-mesenchymal plasticity with immune subtype analysis to understand the relationship between epithelial plasticity and immune subtype across cancers. We find heterogeneity of epithelial-mesenchymal transition (EMT) status both within and between cancer types, with greater heterogeneity in the expression of EMT-related factors than of MET-related factors. We also find that specific immune subtypes have associated EMT scores and differential expression of immune checkpoint markers.