scholarly journals Keep it simple - A case study of model development in the context of the Dynamic Stocks and Flows (DSF) task

2010 ◽  
Vol 2 (2) ◽  
pp. 38-51 ◽  
Author(s):  
Marc Halbrügge
Keyword(s):  
Goodness Of Fit ◽  
Model Development ◽  
Cognitive Model ◽  
Training Data ◽  
Sequence Matching ◽  
Data Set ◽  
Depth Analysis ◽  
Stocks And Flows ◽  
Matching Techniques

Keep it simple - A case study of model development in the context of the Dynamic Stocks and Flows (DSF) taskThis paper describes the creation of a cognitive model submitted to the ‘Dynamic Stocks and Flows’ (DSF) modeling challenge. This challenge aims at comparing computational cognitive models for human behavior during an open ended control task. Participants in the modeling competition were provided with a simulation environment and training data for benchmarking their models while the actual specification of the competition task was withheld. To meet this challenge, the cognitive model described here was designed and optimized for generalizability. Only two simple assumptions about human problem solving were used to explain the empirical findings of the training data. In-depth analysis of the data set prior to the development of the model led to the dismissal of correlations or other parametric statistics as goodness-of-fit indicators. A new statistical measurement based on rank orders and sequence matching techniques is being proposed instead. This measurement, when being applied to the human sample, also identifies clusters of subjects that use different strategies for the task. The acceptability of the fits achieved by the model is verified using permutation tests.

scholarly journals Opportunities and Constraints in Applying Artificial Neural Networks (ANNs) in Food Authentication. Honey—A Case Study

2021 ◽  
Vol 11 (15) ◽  
pp. 6723
Author(s):  
Ariana Raluca Hategan ◽  
Romulus Puscas ◽  
Gabriela Cristea ◽  
Adriana Dehelean ◽  
Francois Guyon ◽  
...  
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Training Sample ◽  
Training Data ◽  
Elemental Content ◽  
Neural Structure ◽  
Food Authentication ◽  
Data Set ◽  
Artificial Neural

The present work aims to test the potential of the application of Artificial Neural Networks (ANNs) for food authentication. For this purpose, honey was chosen as the working matrix. The samples were originated from two countries: Romania (50) and France (53), having as floral origins: acacia, linden, honeydew, colza, galium verum, coriander, sunflower, thyme, raspberry, lavender and chestnut. The ANNs were built on the isotope and elemental content of the investigated honey samples. This approach conducted to the development of a prediction model for geographical recognition with an accuracy of 96%. Alongside this work, distinct models were developed and tested, with the aim of identifying the most suitable configurations for this application. In this regard, improvements have been continuously performed; the most important of them consisted in overcoming the unwanted phenomenon of over-fitting, observed for the training data set. This was achieved by identifying appropriate values for the number of iterations over the training data and for the size and number of the hidden layers and by introducing of a dropout layer in the configuration of the neural structure. As a conclusion, ANNs can be successfully applied in food authenticity control, but with a degree of caution with respect to the “over optimization” of the correct classification percentage for the training sample set, which can lead to an over-fitted model.

scholarly journals Using Model Performance to Assess the Representativeness of Data for Model Development and Calibration in Financial Institutions

Risks ◽  
2021 ◽  
Vol 9 (11) ◽  
pp. 204
Author(s):  
Chamay Kruger ◽  
Willem Daniel Schutte ◽  
Tanja Verster
Keyword(s):  
Case Studies ◽  
Model Development ◽  
Model Performance ◽  
Data Set ◽  
Pooled Data ◽  
External Data ◽  
First Case ◽  
Multivariate Prediction ◽  
Credit Data

This paper proposes a methodology that utilises model performance as a metric to assess the representativeness of external or pooled data when it is used by banks in regulatory model development and calibration. There is currently no formal methodology to assess representativeness. The paper provides a review of existing regulatory literature on the requirements of assessing representativeness and emphasises that both qualitative and quantitative aspects need to be considered. We present a novel methodology and apply it to two case studies. We compared our methodology with the Multivariate Prediction Accuracy Index. The first case study investigates whether a pooled data source from Global Credit Data (GCD) is representative when considering the enrichment of internal data with pooled data in the development of a regulatory loss given default (LGD) model. The second case study differs from the first by illustrating which other countries in the pooled data set could be representative when enriching internal data during the development of a LGD model. Using these case studies as examples, our proposed methodology provides users with a generalised framework to identify subsets of the external data that are representative of their Country’s or bank’s data, making the results general and universally applicable.

Artificial intelligence (AI) to improve patient outcomes in community oncology practices.

2019 ◽  
Vol 37 (15_suppl) ◽  
pp. e18098-e18098
Author(s):  
John Frownfelter ◽  
Sibel Blau ◽  
Ray D. Page ◽  
John Showalter ◽  
Kelly Miller ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Goodness Of Fit ◽  
Predictive Analytics ◽  
Training Data ◽  
Model Specification ◽  
Care Plan ◽  
Cancer Type ◽  
Pilot Program ◽  
Data Set ◽  
Ecog Ps

e18098 Background: Artificial Intelligence(AI) for predictive analytics has been studied extensively in diagnostic imaging and genetic testing. Cognitive analytics adds by suggesting interventions that optimize health outcomes using real-time data and machine learning. Herein, we report the results of a pilot study of the Jvion, Inc. Cognitive Clinical Success Machine (CCSM), an eigen vector-based deep learning AI technology. Methods: The CCSM uses electronic medical record (EMR) and publicly available socioeconomic/behavioral databases to create a n-dimensional space within which patients are mapped along vectors resulting in thousands of relevant clusters of clinically/behaviorally similar patients. These clusters have a mathematical propensity to respond to a clinical intervention which are updated dynamically with new data from the site. The CCSM generates recommendations for the provider to consider as they develop a care plan based on the patients’ cluster. We tested and trained the CCSM technology at 3 US oncology practices for the risk (low, intermediate, high) of 4 specific outcomes: 30 day severe pain, 30 day mortality, 6 month clinical deterioration (ECOG-PS), and 6 month diagnosis of major depressive disorder (MDD). We report the accuracy of the CCSM based on the testing and training data sets. Area under the curve (AUC) was calculated to show goodness of fit of classification models for each outcome. Results: In the training/testing data set there were 371,787 patients from the 3 sites: female = 61.3%; age ≤ 50 = 21.3%, 51-65 = 26.9%, > 65 = 51.9%; white/Caucasian = 43.4%, black/African American = 5.9%, unknown race = 43.4%. Cancer types were unknown/missing for 66.3% of patients and stage for 90.4% of patients. AUC range per vector: 30 day severe/recurrent pain = 0.85-0.90; 30-day mortality = 0.86-0.97; 6-month ECOG-PS decline of 1 point = 0.88-0.92; and 6-month diagnosis of MDD = 0.77-0.90. Conclusions: The high AUC indicates good separation between true positives/negatives (proper model specification for classifying the risk of each outcome) regardless of the degree of missing data for variables including cancer type and stage. Following testing, a 6 month pilot program was implemented (06/2018-11/2018). Final results of the pilot program are pending.

scholarly journals Predictive Performance Analysis of PDF – IDF Model Types Using Rainfall Observations from Fourteen Gauged Stations

2021 ◽  
pp. 125-143
Author(s):  
Ify L. Nwaogazie ◽  
M. G. Sam ◽  
A. O. David
Keyword(s):  
Return Period ◽  
Flood Control ◽  
Goodness Of Fit ◽  
Rainfall Intensity ◽  
Model Development ◽  
Predictive Performance ◽  
Specific Model ◽  
Data Set ◽  
Pearson Type ◽  
Significant Difference

The design of structures for flood mitigation depends on the adequate estimation of rainfall intensity over a given catchment which is achieved by the rainfall intensity duration frequency modelling. In this study, an extensive comparative analyses were carried out on the predictive performance of three PDF – IDF model types, namely: Gumbel Extreme Value Type 1 (GEVT – 1), Log-Pearson Type 3 (LPT – 3) and Normal Distribution (ND) in 14 selected cities in Southern Nigeria. This is to rank the order of best performance. The principle of general model development was adopted in which rainfall intensities at different durations and specified return periods were used as input data set. This is not same as return period specific model that involves rainfall intensities for various durations and a given return period. The predicted rainfall intensity values with the PDF – IDF model types indicate high goodness of fit (R2) and Mean Squared Errors (MSE) ranging from: (a) R2 = 0.875 – 0.992; MSE = 33.17 – 224.6 for GEVT – 1; (b) R2 = 0.849 – 0.990; MSE = 65.34 – 405.5 for LPT – 3 and (c) R2 = 0.839 – 0.992; MSE = 29.23 – 200.2 for ND. The comparative analysis of all the 42 general models (14 locations versus 3 model types) considered showed that the order of best performance is LPT – 3 1st, GEVT - 1 2nd and ND 3rd for each return period (10, 50 and 100 years). The Kruskal Wallis test of significance indicates that no significant difference exists in the predictive performance of the three General models across the board. This may be due to the fact that the fourteen locations of the study area are bordering with the Atlantic Ocean and seems to have similar climatology. These developed General models are recommended for the computation of intensities in the fourteen locations for the design of flood control structures; and the order of preference should be LPT – 3 > GEVT – 1 > ND.

scholarly journals 3D Design Using Generative Adversarial Networks and Physics-Based Validation

2019 ◽  
Vol 142 (7) ◽  
Author(s):  
Dule Shu ◽  
James Cunningham ◽  
Gary Stump ◽  
Simon W. Miller ◽  
Michael A. Yukish ◽  
...  
Keyword(s):  
Evaluation Process ◽  
3D Models ◽  
Training Data ◽  
Generative Adversarial Networks ◽  
3D Design ◽  
Generative Adversarial Network ◽  
Data Set ◽  
High Performing ◽  
Adversarial Network

Abstract The authors present a generative adversarial network (GAN) model that demonstrates how to generate 3D models in their native format so that they can be either evaluated using complex simulation environments or realized using methods such as additive manufacturing. Once initially trained, the GAN can create additional training data itself by generating new designs, evaluating them in a physics-based virtual environment, and adding the high performing ones to the training set. A case study involving a GAN model that is initially trained on 4045 3D aircraft models is used for demonstration, where a training data set that has been updated with GAN-generated and evaluated designs results in enhanced model generation, in both the geometric feasibility and performance of the designs. Z-tests on the performance scores of the generated aircraft models indicate a statistically significant improvement in the functionality of the generated models after three iterations of the training-evaluation process. In the case study, a number of techniques are explored to structure the generate-evaluate process in order to balance the need to generate feasible designs with the need for innovative designs.

scholarly journals Compositionality Decomposed: How do Neural Networks Generalise?

2020 ◽  
Vol 67 ◽  
pp. 757-795
Author(s):  
Dieuwke Hupkes ◽  
Verna Dankers ◽  
Mathijs Mul ◽  
Elia Bruni
Keyword(s):  
Neural Networks ◽  
Compositional Data ◽  
Empirical Studies ◽  
Neural Model ◽  
Training Data ◽  
Data Set ◽  
Depth Analysis ◽  
The One ◽  
Transformer Model ◽  
Composition Operations

Despite a multitude of empirical studies, little consensus exists on whether neural networks are able to generalise compositionally, a controversy that, in part, stems from a lack of agreement about what it means for a neural model to be compositional. As a response to this controversy, we present a set of tests that provide a bridge between, on the one hand, the vast amount of linguistic and philosophical theory about compositionality of language and, on the other, the successful neural models of language. We collect different interpretations of compositionality and translate them into five theoretically grounded tests for models that are formulated on a task-independent level. In particular, we provide tests to investigate (i) if models systematically recombine known parts and rules (ii) if models can extend their predictions beyond the length they have seen in the training data (iii) if models’ composition operations are local or global (iv) if models’ predictions are robust to synonym substitutions and (v) if models favour rules or exceptions during training. To demonstrate the usefulness of this evaluation paradigm, we instantiate these five tests on a highly compositional data set which we dub PCFG SET and apply the resulting tests to three popular sequence-to-sequence models: a recurrent, a convolution-based and a transformer model. We provide an in-depth analysis of the results, which uncover the strengths and weaknesses of these three architectures and point to potential areas of improvement.

scholarly journals Use of artificial intelligence to estimate population health indicators in France

2020 ◽  
Vol 30 (Supplement_5) ◽  
Author(s):  
R Haneef ◽  
S Fuentes ◽  
R Hrzic ◽  
S Fosse-Edorh ◽  
S Kab ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Machine Learning ◽  
Training Data ◽  
Supervised Machine Learning ◽  
Data Sets ◽  
Data Set ◽  
Selection Of

Abstract Background The use of artificial intelligence is increasing to estimate and predict health outcomes from large data sets. The main objectives were to develop two algorithms using machine learning techniques to identify new cases of diabetes (case study I) and to classify type 1 and type 2 (case study II) in France. Methods We selected the training data set from a cohort study linked with French national Health database (i.e., SNDS). Two final datasets were used to achieve each objective. A supervised machine learning method including eight following steps was developed: the selection of the data set, case definition, coding and standardization of variables, split data into training and test data sets, variable selection, training, validation and selection of the model. We planned to apply the trained models on the SNDS to estimate the incidence of diabetes and the prevalence of type 1/2 diabetes. Results For the case study I, 23/3468 and for case study II, 14/3481 SNDS variables were selected based on an optimal balance between variance explained and using the ReliefExp algorithm. We trained four models using different classification algorithms on the training data set. The Linear Discriminant Analysis model performed best in both case studies. The models were assessed on the test datasets and achieved a specificity of 67% and a sensitivity of 62% in case study I, and a specificity of 97 % and sensitivity of 100% in case study II. The case study II model was applied to the SNDS and estimated the prevalence of type 1 diabetes in 2016 in France of 0.3% and for type 2, 4.4%. The case study model I was not applied to the SNDS. Conclusions The case study II model to estimate the prevalence of type 1/2 diabetes has good performance and will be used in routine surveillance. The case study I model to identify new cases of diabetes showed a poor performance due to missing necessary information on determinants of diabetes and will need to be improved for further research.

scholarly journals Representing Small Commercial Building Faults in EnergyPlus, Part I: Model Development

Buildings ◽  
2019 ◽  
Vol 9 (11) ◽  
pp. 233 ◽  
Author(s):  
Janghyun Kim ◽  
Stephen Frank ◽  
James E. Braun ◽  
David Goldwasser
Keyword(s):  
Model Development ◽  
Cost Effective ◽  
Research Area ◽  
Commercial Buildings ◽  
Fault Detection And Diagnosis ◽  
Machine Learning Algorithms ◽  
Training Data ◽  
Fault Models ◽  
Commercial Building ◽  
Data Set

Small commercial buildings (those with less than approximately 1000 m2 of total floor area) often do not have access to cost-effective automated fault detection and diagnosis (AFDD) tools for maintaining efficient building operations. AFDD tools based on machine-learning algorithms hold promise for lowering cost barriers for AFDD in small commercial buildings; however, such algorithms require access to high-quality training data that is often difficult to obtain. To fill the gap in this research area, this study covers the development (Part I) and validation (Part II) of fault models that can be used with the building energy modeling software EnergyPlus® and OpenStudio® to generate a cost-effective training data set for developing AFDD algorithms. Part I (this paper) presents a library of fault models, including detailed descriptions of each fault model structure and their implementation with EnergyPlus. This paper also discusses a case study of training data set generation, representing an actual building.

scholarly journals Development and validation of a predictive model for 90-day readmission following elective spine surgery

2018 ◽  
Vol 29 (3) ◽  
pp. 327-331 ◽  
Author(s):  
Scott L. Parker ◽  
Ahilan Sivaganesan ◽  
Silky Chotai ◽  
Matthew J. McGirt ◽  
Anthony L. Asher ◽  
...  
Keyword(s):  
Predictive Model ◽  
Model Validation ◽  
Spine Surgery ◽  
Model Development ◽  
Cost Of Care ◽  
Training Data ◽  
Data Set ◽  
C Statistic ◽  
Increased Risk ◽  
Prospective Longitudinal

OBJECTIVEHospital readmissions lead to a significant increase in the total cost of care in patients undergoing elective spine surgery. Understanding factors associated with an increased risk of postoperative readmission could facilitate a reduction in such occurrences. The aims of this study were to develop and validate a predictive model for 90-day hospital readmission following elective spine surgery.METHODSAll patients undergoing elective spine surgery for degenerative disease were enrolled in a prospective longitudinal registry. All 90-day readmissions were prospectively recorded. For predictive modeling, all covariates were selected by choosing those variables that were significantly associated with readmission and by incorporating other relevant variables based on clinical intuition and the Akaike information criterion. Eighty percent of the sample was randomly selected for model development and 20% for model validation. Multiple logistic regression analysis was performed with Bayesian model averaging (BMA) to model the odds of 90-day readmission. Goodness of fit was assessed via the C-statistic, that is, the area under the receiver operating characteristic curve (AUC), using the training data set. Discrimination (predictive performance) was assessed using the C-statistic, as applied to the 20% validation data set.RESULTSA total of 2803 consecutive patients were enrolled in the registry, and their data were analyzed for this study. Of this cohort, 227 (8.1%) patients were readmitted to the hospital (for any cause) within 90 days postoperatively. Variables significantly associated with an increased risk of readmission were as follows (OR [95% CI]): lumbar surgery 1.8 [1.1–2.8], government-issued insurance 2.0 [1.4–3.0], hypertension 2.1 [1.4–3.3], prior myocardial infarction 2.2 [1.2–3.8], diabetes 2.5 [1.7–3.7], and coagulation disorder 3.1 [1.6–5.8]. These variables, in addition to others determined a priori to be clinically relevant, comprised 32 inputs in the predictive model constructed using BMA. The AUC value for the training data set was 0.77 for model development and 0.76 for model validation.CONCLUSIONSIdentification of high-risk patients is feasible with the novel predictive model presented herein. Appropriate allocation of resources to reduce the postoperative incidence of readmission may reduce the readmission rate and the associated health care costs.

scholarly journals Peer groups for organisational learning: Clustering with practical constraints

PLoS ONE ◽  
2021 ◽  
Vol 16 (6) ◽  
pp. e0251723
Author(s):  
Daniel W. Kennedy ◽  
Jessica Cameron ◽  
Paul P. -Y. Wu ◽  
Kerrie Mengersen
Keyword(s):  
Goodness Of Fit ◽  
Peer Group ◽  
Negative Relationship ◽  
Peer Groups ◽  
Organisational Learning ◽  
High Dimensional ◽  
Data Set ◽  
Statistical Knowledge ◽  
Distinguishing Features

Peer-grouping is used in many sectors for organisational learning, policy implementation, and benchmarking. Clustering provides a statistical, data-driven method for constructing meaningful peer groups, but peer groups must be compatible with business constraints such as size and stability considerations. Additionally, statistical peer groups are constructed from many different variables, and can be difficult to understand, especially for non-statistical audiences. We developed methodology to apply business constraints to clustering solutions and allow the decision-maker to choose the balance between statistical goodness-of-fit and conformity to business constraints. Several tools were utilised to identify complex distinguishing features in peer groups, and a number of visualisations are developed to explain high-dimensional clusters for non-statistical audiences. In a case study where peer group size was required to be small (≤ 100 members), we applied constrained clustering to a noisy high-dimensional data-set over two subsequent years, ensuring that the clusters were sufficiently stable between years. Our approach not only satisfied clustering constraints on the test data, but maintained an almost monotonic negative relationship between goodness-of-fit and stability between subsequent years. We demonstrated in the context of the case study how distinguishing features between clusters can be communicated clearly to different stakeholders with substantial and limited statistical knowledge.

Sign in / Sign up

Export Citation Format

Share Document