A Comparison between Term-Independence Retrieval Models for Ad Hoc Retrieval

In Information Retrieval, numerous retrieval models or document ranking functions have been developed in the quest for better retrieval effectiveness. Apart from some formal retrieval models formulated on a theoretical basis, various recent works have applied heuristic constraints to guide the derivation of document ranking functions. While many recent methods are shown to improve over established and successful models, comparison among these new methods under a common environment is often missing. To address this issue, we perform an extensive and up-to-date comparison of leading term-independence retrieval models implemented in our own retrieval system. Our study focuses on the following questions: (RQ1) Is there a retrieval model that consistently outperforms all other models across multiple collections; (RQ2) What are the important features of an effective document ranking function? Our retrieval experiments performed on several TREC test collections of a wide range of sizes (up to the terabyte-sized Clueweb09 Category B) enable us to answer these research questions. This work also serves as a reproducibility study for leading retrieval models. While our experiments show that no single retrieval model outperforms all others across all tested collections, some recent retrieval models, such as MATF and MVD, consistently perform better than the common baselines.

A Cross-Sectional Reproducibility Study of a Standard Camera Sensor Using Artificial Intelligence to Assess Food Items: The FoodIntech Project

Having a system to measure food consumption is important to establish whether individual nutritional needs are being met in order to act quickly and to minimize the risk of undernutrition. Here, we tested a smartphone-based food consumption assessment system named FoodIntech. FoodIntech, which is based on AI using deep neural networks (DNN), automatically recognizes food items and dishes and calculates food leftovers using an image-based approach, i.e., it does not require human intervention to assess food consumption. This method uses one-input and one-output images by means of the detection and synchronization of a QRcode located on the meal tray. The DNN are then used to process the images and implement food detection, segmentation and recognition. Overall, 22,544 situations analyzed from 149 dishes were used to test the reliability of this method. The reliability of the AI results, based on the central intra-class correlation coefficient values, appeared to be excellent for 39% of the dishes (n = 58 dishes) and good for 19% (n = 28). The implementation of this method is an effective way to improve the recognition of dishes and it is possible, with a sufficient number of photos, to extend the capabilities of the tool to new dishes and foods.

Reproducibility and feasibility of an online self-administered food frequency questionnaire for use among adult Norwegians

Background: New methods of dietary assessment are increasingly making use of online technologies. The development of a new online food frequency questionnaire warranted investigation of its feasibility and the reproducibility of its results. Objective: To investigate the feasibility and reproducibility of a newly developed online FFQ (WebFFQ). Design: The semiquantitative WebFFQ was designed to assess the habitual diet the previous year, with questions about frequency of intake and portion sizes. Estimations of portion sizes include both pictures and household measures, depending on the type of food in question. In two independent cross-sectional studies conducted in 2015 and 2016, adults were recruited by post following random selection from the general population. In the first study, participants (n = 229) filled in the WebFFQ and answered questions about its feasibility, and in two subsequent focus group meetings, participants (n = 9) discussed and gave feedback about the feasibility of the WebFFQ. In the second study, the WebFFQ’s reproducibility was assessed by asking participants (n = 164) to fill it in on two separate occasions, 12 weeks apart. Moreover, in the second study, participants were offered personal dietary feedback, a monetary gift certificate, or both, as incentives to complete the study. Results: In the feasibility study, evaluation form results showed that participants raised issues regarding the estimation of portion size and the intake of seasonal foods as being particularly challenging; furthermore, in the focus group discussions, personal feedback on diet was perceived to be a more motivating factor than monetary reward. In the reproducibility study, total food intake was lower in the second WebFFQ; however, 63% of the food groups were not significantly different from those in the first WebFFQ. Correlations of food intake ranged from 0.62 to 0.90, >86% of the participants were classified into the same or adjacent quartiles, and misclassification ranged from 0 to 3%. Average energy intake was 3.5% lower (p = 0.001), fiber showed the least difference at 1.6% (p = 0.007), and sugar intake differed the most at −6.8% (borderline significant, p = 0.08). Percentage energy obtained from macronutrients did not differ significantly between the first and second WebFFQs. Conclusion: Our results suggest that at group level, the WebFFQ showed good reproducibility for the estimations of intake of food groups, energy, and nutrients. The feasibility of the WebFFQ is good; however, revisions to further improve portion size estimations should be included in future versions. The WebFFQ is considered suitable for dietary assessments for healthy adults in the Norwegian population.

Bedaquiline Drug Resistance Emergence Assessment in MDR-TB (DREAM): a 5-Year Prospective In-Vitro Surveillance Study of Bedaquiline and Other Second-Line Drug-Susceptibility Testing in MDR-TB Isolates

Bedaquiline Drug Resistance Emergence Assessment in Multidrug-resistant-tuberculosis (MDR-TB) (DREAM) was a 5-year (2015–2019) phenotypic drug-resistance surveillance study across 11 countries. DREAM assessed the susceptibility of 5036 MDR-TB isolates of bedaquiline-treatment-naïve patients to bedaquiline and other anti-tuberculosis drugs by the 7H9 broth microdilution (BMD) and 7H10/7H11 agar dilution (AD) minimal inhibitory concentration (MIC) methods. Bedaquiline AD MIC quality control (QC) range for the H37Rv reference strain was unchanged, but the BMD MIC QC range (0.015–0.12 μg/ml) was adjusted compared with ranges from a multilaboratory, multicountry reproducibility study conforming to Clinical and Laboratory Standards Institute Tier-2 criteria. Epidemiological cut-off values of 0.12 μg/ml by BMD and 0.25 μg/ml by AD were consistent with previous bedaquiline breakpoints. An area of technical uncertainty or Intermediate category was set at 0.25 μg/ml and 0.5 μg/ml for BMD and AD, respectively. When applied to the 5036 MDR-TB isolates, bedaquiline-susceptible, intermediate and bedaquiline-resistant rates were 97.9%, 1.5% and 0.6%, respectively, for BMD, and 98.8%, 0.8% and 0.4% for AD. Resistance rates were: ofloxacin 35.1%, levofloxacin 34.2%, moxifloxacin 33.3%, 1.5% linezolid and 2% clofazimine. Phenotypic cross resistance between bedaquiline and clofazimine was 0.4% in MDR-TB and 1% in pre-extensively drug-resistant (pre-XDR-TB)/XDR-TB populations. Co-resistance to bedaquiline and linezolid, and clofazimine and linezolid, were 0.1% and 0.3%, respectively, in MDR-TB, and 0.2% and 0.4% in pre-XDR-TB/XDR-TB populations. Resistance rates to bedaquiline appear to be low in the bedaquiline-treatment-naïve population. No treatment-limiting patterns for cross-resistance and co-resistance have been identified with key TB drugs to date.

Harmonization of Flow Cytometric Minimal Residual Disease Assessment in Multiple Myeloma in Centers of Polish Myeloma Consortium

Minimal residual disease (MRD) status is now considered as one of the most relevant prognostic factors in multiple myeloma (MM) while MRD negativity became an important endpoint in clinical trials. Here, we report the results of the first study evaluating the reproducibility of high-sensitivity flow cytometry MM MRD assessment in four laboratories in Poland. EuroFlow protocols for instrument setting standardization and sample preparation in MM MRD assessment were implemented in each laboratory. In the inter-laboratory reproducibility study, 12 bone marrow samples from MM patients were distributed and processed in participant laboratories. In the inter-operator concordance study, 13 raw data files from MM MRD measurements were analyzed by five independent operators. The inter-laboratory study showed high 95% overall concordance of results among laboratories. In the inter-operator study, 89% of MRD results reported were concordant, and the highest immunophenotype interpretation differences with regard to expression of CD27, CD45, CD81 were noticed. We confirmed the applicability and feasibility of the EuroFlow protocol as a highly sensitive method of MRD evaluation in MM. Results of our inter-center comparison study demonstrate that the standardization of MM MRD assessment protocols is highly desirable to improve quality and comparability of results within and between different clinical trials.

Validating the Knowledge Bank Approach for Personalized Prediction of Survival in Acute Myeloid Leukemia: a Reproducibility Study

Reproducibility is not only essential for the integrity of scientific research, but is also a prerequisite of model validation and refinement for future application of (predictive) algorithms. However, reproducible research is becoming increasingly challenging, particularly in high-dimensional genomic data analyses with complex statistical or algorithmic techniques. Given that there are no mandatory requirements in most biomedical and statistical journals to provide the original data, analytical source code, or other relevant materials for publication, accessibility to these supplements naturally suggests a greater credibility of published work. In this study, we performed a reproducibility assessment of the notable paper by Gerstung et al. published in Nature Genetics (2017) by rerunning the analysis using their original code and data, which are publicly accessible. Despite a perfect open science setting, it was challenging to reproduce the entire research project; reasons included coding errors, suboptimal code legibility, incomplete documentation, intensive computations, and an R computing environment that could no longer be re-established. We learn that availability of code and data does not guarantee transparency and reproducibility of a study; in contrast, the source code is still liable to error and obsolescence, essentially due to methodological complexity, lack of editorial reproducibility checking at submission, and updates of software and operating environment. Building on the experience gained, we propose practical criteria for the conduct and reporting of reproducibility studies for future researchers.

Reproducibility study of the Fabbri et al. 2017 model of the human sinus node action potential

The sinoatrial node (SAN) is the natural pacemaker of the mammalian heart. It has been the subject of several mathematical studies, aimed at reproducing its electrical response under normal sinus rhythms, as well as under various conditions. Such studies were traditionally done using data from rabbit SAN cells. More recently, human SAN cell data have become available, resulting in the publication of a human SAN cell model (Fabbri et al., 2017), along with its CellML version. Here, we used the CellML file provided by the model authors, together with some SED-ML files and Python scripts that we created to reproduce the main results of the aforementioned modeling study. EDITOR'S NOTE (v2): this article and its OMEX archive are republished with technical changes made to the corresponding Python scripts to remove a run-time error message displayed when executing each simulation.