scholarly journals A sampling approach to Debiasing the offline evaluation of recommender systems

Author(s):  
Diego Carraro ◽  
Derek Bridge

AbstractOffline evaluation of recommender systems (RSs) mostly relies on historical data, which is often biased. The bias is a result of many confounders that affect the data collection process. In such biased data, user-item interactions are Missing Not At Random (MNAR). Measures of recommender system performance on MNAR test data are unlikely to be reliable indicators of real-world performance unless something is done to mitigate the bias. One widespread way that researchers try to obtain less biased offline evaluation is by designing new, supposedly unbiased performance metrics for use on MNAR test data. We investigate an alternative solution, a sampling approach. The general idea is to use a sampling strategy on MNAR data to generate an intervened test set with less bias — one in which interactions are Missing At Random (MAR) or, at least, one that is more MAR-like. An existing example of this approach is SKEW, a sampling strategy that aims to adjust for the confounding effect that an item’s popularity has on its likelihood of being observed. In this paper, after extensively surveying the literature on the bias problem in the offline evaluation of RSs, we propose and formulate a novel sampling approach, which we call WTD; we also propose a more practical variant, which we call WTD_H. We compare our methods to SKEW and to two baselines which perform a random intervention on MNAR data. We empirically validate for the first time the effectiveness of SKEW and we show our approach to be a better estimator of the performance that one would obtain on (unbiased) MAR test data. Our strategy benefits from high generality (e.g. it can also be employed for training a recommender) and low overheads (e.g. it does not require any learning).

2021 ◽  
Vol 37 (3) ◽  
pp. 655-671
Author(s):  
Paolo Righi ◽  
Piero Demetrio Falorsi ◽  
Stefano Daddi ◽  
Epifania Fiorello ◽  
Pierpaolo Massoli ◽  
...  

Abstract For the first time in 2018 the Italian Institute of Statistics (Istat) implemented the annual Permanent Population Census which relies on the Population Base Register (PBR) and the Population Coverage Survey (PCS). This article provides a general overview of the PCS sampling design, which makes use of the PBR to correct population counts with the extended dual system estimator (Nirel and Glickman 2009). The sample allocation, proven optimal under a set of precision constraints, is based on preliminary estimates of individual probabilities of over-coverage and under-coverage. It defines the expected sample size in terms of individuals, and it oversamples the sub-populations subject to the risk of under/over coverage. Finally, the article introduces a sample selection method, which to the greatest extent possible satisfies the planned allocation of persons in terms of socio-demographic characteristics. Under acceptable assumptions, the article also shows that the sampling strategy enhances the precision of the estimates.


Blood ◽  
2016 ◽  
Vol 128 (22) ◽  
pp. 3652-3652 ◽  
Author(s):  
Patrick T. McGann ◽  
Min Dong ◽  
Anu Marahatta ◽  
Thad A. Howard ◽  
Tomoyuki Mizuno ◽  
...  

Abstract Background: Hydroxyurea is the primary disease modifying therapy for patients with sickle cell anemia (SCA). The clinical and laboratory benefits of hydroxyurea are the greatest when escalated to the maximum tolerated dose (MTD). The process of dose escalation to MTD requires expertise and can be tedious, often taking 6-12 months to titrate to the optimal dose. In addition, due in part to inter-patient variability in hydroxyurea pharmacokinetics (PK), the MTD varies among patients with a range of 15-35 mg/kg/day. We utilized a population PK model in combination with Bayesian estimation and a sparse sampling strategy, to individualize dosing of children starting hydroxyurea treatment. Methods: The Therapeutic Response Evaluation and Adherence Trial (TREAT, ClinicalTrials.gov NCT02286154) is a prospective study of hydroxyurea for children with SCA. The primary objective is to develop and evaluate a population PK-based model to predict hydroxyurea MTD through an individualized dosing strategy. A sparse sampling approach was developed to allow practical sampling from young children with SCA. The sampling strategy includes administering a single oral 20 mg/kg dose followed by collection of small quantities of blood (~100uL) at three post-dosing time points (15-20 minutes, 50-60 minutes, and ~3 hours). Baseline labs (including liver and renal function) are typically collected by venipuncture, while the other two samples are drawn by fingerstick or heelstick. Plasma hydroxyurea concentrations are measured using HPLC using an internal standard of methylurea. Using the population PK model with Bayesian estimation and hydroxyurea concentrations measured at the three specified time points, hydroxyurea exposure is estimated using specialized therapeutic drug monitoring software (MWPharm, Mediware, Prague, Czech Republic). Using the area under the curve (AUC0-inf) estimated by the model, we calculate a starting dose that is predicted to achieve an AUC of 115 ug*h/mL, which was the mean AUC value at MTD for a large cohort of children from a previous study (Dong M et al. Br J Clin Pharmacol 2016). The primary objective is to select a starting dose that is close to actual MTD, to reduce the time to maximum therapeutic effect and need for dose modifications before achieving MTD. Results: From December 2014 through June 2016, 20 children taking taking hydroxyurea for the first time were enrolled in TREAT. Seventeen of the 20 participants had all 3 post-treatment PK samples collected and processed to allow calculation of an individualized PK-based dose, while 3 had difficulties in sampling or processing that prevented a safe PK-guided dose recommendation. These 3 participants were started at the standard hydroxyurea dose of 20 mg/kg/day. The Table summarizes baseline characteristics for the initial study population, notable for a very young starting age with 13/20 (65%) less than two years of age. Twelve children with PK-based initial dosing have been treated with hydroxyurea for at least six months. Despite the young starting age, after six months of hydroxyurea, children have documented increases in total Hb (1.4+/-1.9 g/dL), HbF (11.3+/-6.4%), and MCV (15+/-8 fL) and decreases in absolute reticulocyte count (-217+/-128 x 109/L) and absolute neutrophil count (-1.0+/-1.9 x 109/L). In 9 of 12 participants, the PK-guided initial dose remained the best clinical dose at six months without significant dose changes except for minor adjustments for weight. Two patients required a single dose escalation due to inadequate marrow suppression, while one required a dose hold and decrease due to neutropenia during and following a viral infection. Conclusions: These data demonstrate that a sparse sampling approach, requiring only 3 blood samples over 3 hours, is able to accurately estimate hydroxyurea exposure in children with SCA. Hydroxyurea exposure, as defined by AUC, was similar with this sparse sampling approach as previous studies that relied upon a more standard and prolonged PK sampling approach. This population PK model is then able to predict a safe starting dose of hydroxyurea that approximates the actual MTD, with clinically significant improvements in laboratory parameters following six months of therapy. This individualized PK-guided dosing regimen should simplify hydroxyurea dosing and reduce the time interval to reach MTD and maximal clinical benefits. Table Table. Disclosures Kalfa: Baxter/Baxalta/Shire: Research Funding. Quinn:Silver Lake Research Corporation: Consultancy; Amgen: Research Funding; Eli Lilly: Research Funding. Ware:Nova Laboratories: Consultancy; Addmedica: Research Funding; Global Blood Therapeutics: Consultancy; Bayer Pharmaceuticals: Consultancy; Biomedomics: Research Funding; Bristol Myers Squibb: Research Funding.


2011 ◽  
Vol 7 (4) ◽  
pp. 47-64 ◽  
Author(s):  
Toly Chen

This paper presents a dynamically optimized fluctuation smoothing rule to improve the performance of scheduling jobs in a wafer fabrication factory. The rule has been modified from the four-factor bi-criteria nonlinear fluctuation smoothing (4f-biNFS) rule, by dynamically adjusting factors. Some properties of the dynamically optimized fluctuation smoothing rule were also discussed theoretically. In addition, production simulation was also applied to generate some test data for evaluating the effectiveness of the proposed methodology. According to the experimental results, the proposed methodology was better than some existing approaches to reduce the average cycle time and cycle time standard deviation. The results also showed that it was possible to improve the performance of one without sacrificing the other performance metrics.


2019 ◽  
Vol 37 (1) ◽  
pp. 76-97 ◽  
Author(s):  
May Nagy ◽  
Dag Bennett ◽  
Charles Graham

Purpose The purpose of this paper is to test the premise that brand growth can come from targeting the poorest consumers at the bottom of the economic pyramid (BOP). This study is the first that uses quantitative marketplace data covering BOP consumer purchase records. Design/methodology/approach The study uses newly available panel data from Egypt covering 15 months and 35 categories of frequently bought consumer goods. Brand penetration rates for socio-demographic tiers are established to explore brand purchasing. The metrics are: penetration, the number of buyers a brand has; and loyalty as measured by purchase frequency and share of category requirements. Findings Buyer behaviour patterns for the poorest consumers do not differ much from those in advanced economies; all brand performance metrics vary according to brand penetration – a double jeopardy effect, and the biggest brands are those that target the whole market, including the base. Research limitations/implications Data are from one country only and while the results confirm that patterns of brand buying in this BOP segment are like those in other markets, more research needs to be done to confirm the finding. Practical implications The biggest brands are those with the most customers, even if those customers are poor and do not buy very often. Growth can therefore be based on marketing interventions that appeal to the largest possible customer base. Social implications There are 2bn BOP consumers worldwide. This research shows that they may already be marginal members of modern economies and consumer culture. Originality/value This paper extends previous research on brand buying behaviour for the first time to the vast base of poor consumers who make up around half of the world’s population. This research shows that strategic approaches that emphasise increasing penetration are most likely to result in brand growth.


Author(s):  
V Murugesan ◽  
Sreejith Plappillimadam ◽  
VJ Saji ◽  
SS Maruthi ◽  
AK Anilkumar

Reliability is one of the critical design parameters for the launch vehicles and its systems. When the systems are ready to fly the first time, only limited test data are available and accordingly reliability assessed will be very low. However, in most cases, the new systems are derived and developed using the knowledge and experience gained from the heritage systems to meet the fresh challenges. Hence, the reliability assessed with the minimum number of tests done on the new system does not truly reflect the inherent reliability of the system. In this paper, an approach and a new analytical model are developed for the reliability assessment of systems with limited test data, giving an accurate weighting for the tests and flight experiences with similar systems. The method gives a systematic procedure for arriving at the weighting factor for test data of the pedigree system, with due consideration of the similarities between the systems and various factors influencing system reliability. The method is illustrated with a case study of a newly developed liquid propellant rocket system. The model is validated using the available test and flight data of two propulsion systems with adequate flight experience. The analytical model is generic in nature and can be applied to reliability analysis of any system, which has considerable similarities with a pedigree system.


Author(s):  
BOJAN CUKIC ◽  
BRIAN J. TAYLOR ◽  
HARSHINDER SINGH

Automated generation of test cases is a prerequisite for fast testing. Whereas the research in automated test data generation addressed the creation of individual test points, test trajectory generation has attracted limited attention. In simple terms, a test trajectory is defined as a series of data points, with each (possibly multidimensional) point relying upon the value(s) of previous point(s). Many embedded systems use data trajectories as inputs, including closed-loop process controllers, robotic manipulators, nuclear monitoring systems, and flight control systems. For these systems, testers can either handcraft test trajectories, use input trajectories from older versions of the system or, perhaps, collect test data in a high fidelity system simulator. While these are valid approaches, they are expensive and time-consuming, especially if the assessment goals require many tests. We developed a framework for expanding a small, conventionally developed set of test trajectories into a large set suitable, for example, for system safety assurance. Statistical regression is the core of this framework. The regression analysis builds a relationship between controllable independent variables and closely correlated dependent variables, which represent test trajectories. By perturbing the independent variables, new test trajectories are generated automatically. Our approach has been applied in the safety assessment of a fault tolerant flight control system. Linear regression, multiple linear regression, and autoregressive techniques are compared. The performance metrics include the speed of test generation and the percentage of "acceptable" trajectories, measured by the domain specific reasonableness checks.


Author(s):  
Jürgen Dunkel ◽  
Ramón Hermoso

AbstractNowadays, most recommender systems are based on a centralized architecture, which can cause crucial issues in terms of trust, privacy, dependability, and costs. In this paper, we propose a decentralized and distributed MANET-based (Mobile Ad-hoc NETwork) recommender system for open facilities. The system is based on mobile devices that collect sensor data about users locations to derive implicit ratings that are used for collaborative filtering recommendations. The mechanisms of deriving ratings and propagating them in a MANET network are discussed in detail. Finally, extensive experiments demonstrate the suitability of the approach in terms of different performance metrics.


Author(s):  
Тхи Хоан Нгуен

В статье впервые дано общее представление о влиянии «Преступления и наказания» Ф. М. Достоевского на творчество вьетнамских писателей на протяжении целого столетия. Автор обращает внимание, что роман «Преступление и наказание» занимает особое место в культурном пространстве Вьетнама, оказывая влияние на поэтику художественных произведений вьетнамских прозаиков, их мировоззренческие и эстетические ориентиры. В работе наглядно иллюстрируется процесс вхождения «Преступления и наказания» в культуру Вьетнама, который шел через творческую адаптацию романа. Автор отмечает, что сложные философские проблемы, которые поднимались в романе Ф. М. Достоевского, вызывали большой интерес у вьетнамских читателей, а у ряда вьетнамских писателей и стремление к подражанию. Исследование показало, что «Преступление и наказание» оказало заметное влияние не только на сюжет, идею художественных произведений, но и на языковой стиль многих известных вьетнамских романистов в ходе модернизации вьетнамской литературы. Интерес к роману великого русского писателя сохраняется сегодня на высоком уровне, что способствует развитию культурных связей между двумя народами. В статье делается вывод о незаменимой позиции романа «Преступление и наказание» в сердцах вьетнамских читателей. Научная новизна настоящего исследования определяется изучением творческой адаптации романа Ф. М. Достоевского в литературной практике вьетнамских художников слова на протяжении XX-XXI веков, а также обозначением различных линий в рецепции известного русского романа во Вьетнаме. Результаты исследования могут быть использованы на занятиях по истории русской и зарубежной литератур, будут интересны учителям-словесникам, а также всем увлеченным художественной литературой и культурой. The article for the first time gives a general idea of the influence of “Crime and Punishment” by F. M. Dostoevsky on the works of Vietnamese writers for a whole century. The author observes that the novel “Crime and Punishment” occupies a special place in the cultural space of Vietnam, influencing the poetics of the artistic works of Vietnamese prose writers, their ideological and aesthetic guidelines. The work clearly illustrates the process by which “Crime and Punishment” enters into the culture of Vietnam, which went through the creative adaptation of the novel. The author notes that the complex philosophical problems that were raised in the novel by F. M. Dostoevsky aroused great interest among Vietnamese readers, and a number of Vietnamese writers also had a desire to imitate. The study showed that “Crime and Punishment” had a noticeable impact not only on the plot, the idea of literary works, but also on the language style of many famous Vietnamese novelists during the modernization of Vietnamese literature. Interest in the novel of the great Russian writer remains today at a high level, which contributes to the development of cultural ties between the two peoples. The article concludes about the irreplaceable position of the novel “Crime and Punishment” in the hearts of Vietnamese readers. The scientific novelty of this study is determined by the study of the creative adaptation of the novel by F. M. Dostoevsky in the literary practice of Vietnamese authors during the 20th-21st centuries, as well as the designation of various lines in the reception of the famous Russian novel in Vietnam. The results of the research can be used in classes on the history of Russian and foreign literature, and will be of interest to teachers of literature as well as to all those who are interested in fiction and culture.


Author(s):  
Toly Chen

This paper presents a dynamically optimized fluctuation smoothing rule to improve the performance of scheduling jobs in a wafer fabrication factory. The rule has been modified from the four-factor bi-criteria nonlinear fluctuation smoothing (4f-biNFS) rule, by dynamically adjusting factors. Some properties of the dynamically optimized fluctuation smoothing rule were also discussed theoretically. In addition, production simulation was also applied to generate some test data for evaluating the effectiveness of the proposed methodology. According to the experimental results, the proposed methodology was better than some existing approaches to reduce the average cycle time and cycle time standard deviation. The results also showed that it was possible to improve the performance of one without sacrificing the other performance metrics.


2020 ◽  
Vol 73 (3) ◽  
pp. 283-288
Author(s):  
А. Seitenova ◽  

In the article, the oeuvre of Sherkhan Murtaza are discussed in the context of the conceptual-figurative character of natural phenomena for the first time. The literary texts which have been previously studied in the context of various aspects have been analyzed in view of a new approach to the study of the artistic role of landscape. The landscape is considered to be a personal view of the world, reproduced by the writer, and in this regard, research along this cognitive line. As exemplified in the novels of “Aisha”, “Black Pearl”, and “Red Arrow” by Sherkhan Murtaza, the parallels of landscape sketches with the author’s intention are analyzed, resulting in uncovering of artistic concepts of earth, sky, fire, and water. A general idea of the concept-forming role of the artistic landscape in the poetry of Sh. Murtaza was systematized and formed.


Sign in / Sign up

Export Citation Format

Share Document