Independent Test Verification: What Metrics Have a Word to Say

Independent Test Verification: Consolidated Experience Report

10.5753/sbqs.2012.15326 ◽

2012 ◽

Author(s):

Nuno Silva ◽

Rui Lopes

Keyword(s):

Team Work ◽

Verification And Validation ◽

Added Value ◽

Large Set ◽

Test Procedures ◽

Opting Out ◽

Independent Verification ◽

Certification Standards ◽

Independent Test ◽

Test Verification

Independent verification and validation (IV&V) has been a key process for decades, and is highlighted in several international certification standards. One of the activities described in the “ESA ISVV Guide” is independent test verification (stated as Integration/Unit Test Procedures and Test Data Verification). This activity is commonly overlooked since customers do not really see the added value of checking thoroughly the validation team work. This article presents the consolidated results of a large set of independent test verifications, including the main difficulties, results obtained and advantages/disadvantages for the industry of these activities. This study will support customers in opting-in or opting-out for this task in future IVV contracts since we provide factual results from some real case studies.

Download Full-text

Using Connectionist Modules for Decision Support

Methods of Information in Medicine ◽

10.1055/s-0038-1634790 ◽

1990 ◽

Vol 29 (03) ◽

pp. 167-181 ◽

Cited By ~ 6

Author(s):

G. Hripcsak

Keyword(s):

Decision Support ◽

Standard Deviation ◽

Confidence Interval ◽

Posterior Probability ◽

Back Propagation ◽

Connectionist Model ◽

Test Set ◽

The Third ◽

Independent Test ◽

Better Than

AbstractA connectionist model for decision support was constructed out of several back-propagation modules. Manifestations serve as input to the model; they may be real-valued, and the confidence in their measurement may be specified. The model produces as its output the posterior probability of disease. The model was trained on 1,000 cases taken from a simulated underlying population with three conditionally independent manifestations. The first manifestation had a linear relationship between value and posterior probability of disease, the second had a stepped relationship, and the third was normally distributed. An independent test set of 30,000 cases showed that the model was better able to estimate the posterior probability of disease (the standard deviation of residuals was 0.046, with a 95% confidence interval of 0.046-0.047) than a model constructed using logistic regression (with a standard deviation of residuals of 0.062, with a 95% confidence interval of 0.062-0.063). The model fitted the normal and stepped manifestations better than the linear one. It accommodated intermediate levels of confidence well.

Download Full-text

Perbedaan Metode Three Compartement Sink dengan Air Panas dan Larutan Klorin terhadap Angka Kuman Alat Makan di RSU Queen Latifa

JURNAL NUTRISIA ◽

10.29238/jnutri.v19i2.27 ◽

2017 ◽

Vol 19 (2) ◽

pp. 126-130

Author(s):

Rifatun Hasanah ◽

Setyowati Setyowati ◽

Noor Tifauzah

Keyword(s):

Infectious Diseases ◽

Confidence Level ◽

Disease Transmission ◽

T Test ◽

Hot Water ◽

Independent Test ◽

The Difference ◽

Hospital Use

Background:One of the efforts in preventing congenital food disease is by washing the cutlery perfectly. The cutlery used by patients with infectious diseases should be noted more, because it has a risk in disease transmission through cutlery. The process of washing the cutlery for infected patients in Queen Latifa Hospital use three compartement sink method with hot water, while the three compartement sink method with clorine solvent has never been tested. Purpose: Research was to determine the difference in the number of germs in the tool was washed using three compartement sink method with hot water and with clorine solvent. Method:Types of research is experiment with rancangan percobaan acak kelompok (RAK). The object of this research is 4 plates and 4 bowls. The number of experimental units in this research were 2 treatments x 2 cutlery x 2 checks x 2 reapetitions = 16 experimental units. The analysis used independent t-test with 95% confidence level. Result :The average number of germs in the cutlery washed using the three compartment sink method with hot water was 1 x 101 cfu / cm2, whereas with chlorine solvent is 0.2 cfu / cm2. Independent test t-test shows p = 0.049 which means the hypothesis is accepted. onclusion : There are differences in the number of germs in the washing cutlery using the three compartment sink method with hot water and with chlorine solvent. Keywords: number of germs, cutlery, three compartment sink

Download Full-text

Perbedaan Metode Three Compartement Sink dengan Air Panas dan Larutan Klorin terhadap Angka Kuman Alat Makan di RSU Queen Latifa

JURNAL NUTRISIA ◽

10.29238/jnutri.v19i2.262 ◽

2017 ◽

Vol 19 (2) ◽

pp. 126

Author(s):

Rifatun Hasanah ◽

Setyowati Setyowati ◽

Noor Tifauzah

Keyword(s):

Infectious Diseases ◽

Confidence Level ◽

Disease Transmission ◽

T Test ◽

Hot Water ◽

Independent Test ◽

The Difference ◽

Hospital Use

Background:One of the efforts in preventing congenital food disease is by washing the cutlery perfectly. The cutlery used by patients with infectious diseases should be noted more, because it has a risk in disease transmission through cutlery. The process of washing the cutlery for infected patients in Queen Latifa Hospital use three compartement sink method with hot water, while the three compartement sink method with clorine solvent has never been tested. Purpose: Research was to determine the difference in the number of germs in the tool was washed using three compartement sink method with hot water and with clorine solvent. Method:Types of research is experiment with rancangan percobaan acak kelompok (RAK). The object of this research is 4 plates and 4 bowls. The number of experimental units in this research were 2 treatments x 2 cutlery x 2 checks x 2 reapetitions = 16 experimental units. The analysis used independent t-test with 95% confidence level. Result :The average number of germs in the cutlery washed using the three compartment sink method with hot water was 1 x 101 cfu / cm2, whereas with chlorine solvent is 0.2 cfu / cm2. Independent test t-test shows p = 0.049 which means the hypothesis is accepted. Conclusion : There are differences in the number of germs in the washing cutlery using the three compartment sink method with hot water and with chlorine solvent.

Download Full-text

Paleomagnetic Studies in the Devonian and Carboniferous Rocks of the Southern Urals: Independent Test of Collision of the Magnitogorsk Island Arc and the Passive Margin of Continent of Laurussia

Доклады Академии наук ◽

10.31857/s086956520003140-7 ◽

2018 ◽

Vol 482 (1) ◽

pp. 64-67

Author(s):

I. Golovanova ◽

◽

K. Danukalov ◽

V. Puchkov ◽

A. Kosarev ◽

...

Keyword(s):

Southern Urals ◽

Island Arc ◽

Passive Margin ◽

The Southern Urals ◽

Independent Test

Download Full-text

The study of acoustic scattering from a single sound-permeable sphere

Multiphase Systems ◽

10.21662/mfs2018.4.012 ◽

2018 ◽

Vol 13 (4) ◽

pp. 79-91 ◽

Cited By ~ 1

Author(s):

E.Sh. Nasibullaeva

Keyword(s):

Speed Of Sound ◽

Numerical Technique ◽

Acoustic Scattering ◽

Computer Time ◽

Physical Parameters ◽

Fast Method ◽

Scattering Field ◽

Permeable Sphere ◽

Test Verification ◽

Good Agreement

The paper presents a generalized mathematical model and numerical investigation of the problem of acoustic scattering from a single sound-permeable sphere during the passage of two types of waves - spherical from a monopole radiation source and a plane one. In solving the Helmholtz equation, a numerical technique based on the fast method of multipoles is used, which allows achieving high accuracy of the results obtained at the lowest cost of computer time. The calculations are compared with known experimental data and a good agreement is obtained. The formulas for calculating the main characteristic of the scattering field (the total scattering cross section) for a sound-permeable sphere are generalized. The effect on this characteristic of the physical parameters of media outside and inside the sphere, such as the density and speed of sound, is shown. A numerical parametric analysis of the pressure distribution around a single sound-permeable sphere for different values of the wave radius, density, and speed of sound of the outer and inner medium of the sphere is carried out. The obtained results will later be used for test verification calculations for the numerical solution of the generalized problem of acoustic scattering of a set of sound-permeable spheres (coaxial or arbitrarily located in space).

Download Full-text

Sex and Test Verification Bias

Circulation ◽

10.1161/01.cir.95.2.405 ◽

1997 ◽

Vol 95 (2) ◽

pp. 405-410 ◽

Cited By ~ 121

Author(s):

Veronique L. Roger ◽

Patricia A. Pellikka ◽

Malcolm R. Bell ◽

Charles W.H. Chow ◽

Kent R. Bailey ◽

...

Keyword(s):

Verification Bias ◽

Test Verification

Download Full-text

How to Assess Prognostic Models for Survival Data: A Case Study in Oncology

Methods of Information in Medicine ◽

10.1055/s-0038-1634384 ◽

2003 ◽

Vol 42 (05) ◽

pp. 564-571 ◽

Cited By ~ 23

Author(s):

M. Schumacher ◽

E. Graf ◽

T. Gerds

Keyword(s):

Test Data ◽

Survival Data ◽

Prediction Error ◽

Classification Scheme ◽

Neural Nets ◽

Brier Score ◽

Data Set ◽

Independent Test ◽

Artificial Neural

Summary Objectives: A lack of generally applicable tools for the assessment of predictions for survival data has to be recognized. Prediction error curves based on the Brier score that have been suggested as a sensible approach are illustrated by means of a case study. Methods: The concept of predictions made in terms of conditional survival probabilities given the patient’s covariates is introduced. Such predictions are derived from various statistical models for survival data including artificial neural networks. The idea of how the prediction error of a prognostic classification scheme can be followed over time is illustrated with the data of two studies on the prognosis of node positive breast cancer patients, one of them serving as an independent test data set. Results and Conclusions: The Brier score as a function of time is shown to be a valuable tool for assessing the predictive performance of prognostic classification schemes for survival data incorporating censored observations. Comparison with the prediction based on the pooled Kaplan Meier estimator yields a benchmark value for any classification scheme incorporating patient’s covariate measurements. The problem of an overoptimistic assessment of prediction error caused by data-driven modelling as it is, for example, done with artificial neural nets can be circumvented by an assessment in an independent test data set.

Download Full-text

Development and validation of a deep learning system to screen vision-threatening conditions in high myopia using optical coherence tomography images

British Journal of Ophthalmology ◽

10.1136/bjophthalmol-2020-317825 ◽

2020 ◽

pp. bjophthalmol-2020-317825

Author(s):

Yonghao Li ◽

Weibo Feng ◽

Xiujuan Zhao ◽

Bingqian Liu ◽

Yan Zhang ◽

...

Keyword(s):

Optical Coherence Tomography ◽

Deep Learning ◽

High Myopia ◽

Large Scale ◽

Learning System ◽

Youden Index ◽

Optical Coherence ◽

Test Dataset ◽

Independent Test ◽

Independent Test Dataset

Background/aimsTo apply deep learning technology to develop an artificial intelligence (AI) system that can identify vision-threatening conditions in high myopia patients based on optical coherence tomography (OCT) macular images.MethodsIn this cross-sectional, prospective study, a total of 5505 qualified OCT macular images obtained from 1048 high myopia patients admitted to Zhongshan Ophthalmic Centre (ZOC) from 2012 to 2017 were selected for the development of the AI system. The independent test dataset included 412 images obtained from 91 high myopia patients recruited at ZOC from January 2019 to May 2019. We adopted the InceptionResnetV2 architecture to train four independent convolutional neural network (CNN) models to identify the following four vision-threatening conditions in high myopia: retinoschisis, macular hole, retinal detachment and pathological myopic choroidal neovascularisation. Focal Loss was used to address class imbalance, and optimal operating thresholds were determined according to the Youden Index.ResultsIn the independent test dataset, the areas under the receiver operating characteristic curves were high for all conditions (0.961 to 0.999). Our AI system achieved sensitivities equal to or even better than those of retina specialists as well as high specificities (greater than 90%). Moreover, our AI system provided a transparent and interpretable diagnosis with heatmaps.ConclusionsWe used OCT macular images for the development of CNN models to identify vision-threatening conditions in high myopia patients. Our models achieved reliable sensitivities and high specificities, comparable to those of retina specialists and may be applied for large-scale high myopia screening and patient follow-up.

Download Full-text