scholarly journals Putting Psychology to the Test: Rethinking Model Evaluation Through Benchmarking and Prediction

2021 ◽  
Vol 4 (3) ◽  
pp. 251524592110268
Author(s):  
Roberta Rocca ◽  
Tal Yarkoni
Keyword(s):  
Machine Learning ◽  
Performance Evaluation ◽  
Model Evaluation ◽  
Evaluation Criteria ◽  
Model Performance ◽  
Scientific Models ◽  
Practical Utility ◽  
Model Performance Evaluation ◽  
Unseen Data ◽  
Reliable Model

Consensus on standards for evaluating models and theories is an integral part of every science. Nonetheless, in psychology, relatively little focus has been placed on defining reliable communal metrics to assess model performance. Evaluation practices are often idiosyncratic and are affected by a number of shortcomings (e.g., failure to assess models’ ability to generalize to unseen data) that make it difficult to discriminate between good and bad models. Drawing inspiration from fields such as machine learning and statistical genetics, we argue in favor of introducing common benchmarks as a means of overcoming the lack of reliable model evaluation criteria currently observed in psychology. We discuss a number of principles benchmarks should satisfy to achieve maximal utility, identify concrete steps the community could take to promote the development of such benchmarks, and address a number of potential pitfalls and concerns that may arise in the course of implementation. We argue that reaching consensus on common evaluation benchmarks will foster cumulative progress in psychology and encourage researchers to place heavier emphasis on the practical utility of scientific models.

scholarly journals EVALUASI PELAKSANAAN HOME VISIT DI SMP

2019 ◽  
Vol 1 (2) ◽  
pp. 111-120
Author(s):  
Bernardinus Agus Arswimba
Keyword(s):  
South Carolina ◽  
Model Evaluation ◽  
Home Visit ◽  
Comprehensive Evaluation ◽  
Evaluation Model ◽  
Evaluation Criteria ◽  
Performance Standards ◽  
Professional Counselor ◽  
The Difference

ABSTRACT Home visit is a responsive service that is urgent, so it needs to be implemented immediately to help students solve problems faced. If home visit is not done well, the problem will have a more complex impact on students. The purpose of the study was to determine the performance of counselors according to or not with the standards and professional counselor evaluation criteria for Comprehensive (Evaluation Model South Carolina) counseling guidance to make decisions or follow-up the programs that have been implemented. The research method used is descriptive evaluation using the Model Evaluation Discrepancy. This model measures the difference between performance standards and real conditions that have been implemented. The instrument of this research is using questionnaires and interviews. The results showed that home visits conducted by counselors at Santa Maria Malang Middle School in the category "close from standard". Keywords: home visit, model evaluation discrepancy

A Bayesian mark–recapture model for multiple-recapture data in a catch-and-release fishery

2009 ◽  
Vol 66 (9) ◽  
pp. 1554-1568 ◽  
Author(s):  
Rebecca Whitlock ◽  
Murdoch McAllister
Keyword(s):  
Model Evaluation ◽  
Bayesian Model ◽  
Evaluation Criteria ◽  
Strong Support ◽  
Fishing Effort ◽  
Length Class ◽  
Fraser River ◽  
Mark Recapture ◽  
Movement Rates ◽  
Seasonal Movement

This paper extends a state–space Bayesian mark–recapture framework to multiple-recapture data to estimate fishery-specific capture and mortality rates and seasonal movement rates for fish in different length classes. The methodology is applied to tag recapture data for white sturgeon ( Acipenser transmontanus ) collected in the recreational fishery and the Canadian Department of Fisheries and Ocean’s test fishery at Albion in the lower Fraser River. Significant differences were found between some estimated movement rates by season and length class, supporting the notion of there being marked differences in seasonal movement patterns between different life history stages of A. transmontanus in the lower Fraser River. Uncertainty in the tag reporting rate parameter, quantified using a recreational creel sampling program, is summarized by a prior distribution. The utility of recreational fishing effort as a model covariate in accounting for seasonal and spatial variation in recapture rates is addressed using Bayesian model evaluation criteria. The data provide strong support in favour of models that include fishing effort as a covariate. The appropriate level of stratification for the recreational catchability parameter q is assessed using Bayesian model evaluation criteria; models in which q is estimated by season and length class have the highest posterior probabilities.

scholarly journals Putting psychology to the test: Rethinking model evaluation through benchmarking and prediction

2020 ◽  
Author(s):  
Roberta Rocca ◽  
Tal Yarkoni
Keyword(s):  
Machine Learning ◽  
Performance Evaluation ◽  
Model Evaluation ◽  
Evaluation Criteria ◽  
Model Performance ◽  
Scientific Models ◽  
Practical Utility ◽  
Model Performance Evaluation ◽  
Unseen Data ◽  
Reliable Model

Consensus on standards for evaluating models and theories is an integral part of every science. Nonetheless, in psychology, relatively little focus has been placed on defining reliable communal metrics to assess model performance. Evaluation practices are often idiosyncratic, and are affected by a number of shortcomings (e.g., failure to assess models' ability to generalize to unseen data) that make it difficult to discriminate between good and bad models. Drawing inspiration from fields like machine learning and statistical genetics, we argue in favor of introducing common benchmarks as a means of overcoming the lack of reliable model evaluation criteria currently observed in psychology. We discuss a number of principles benchmarks should satisfy to achieve maximal utility; identify concrete steps the community could take to promote the development of such benchmarks; and address a number of potential pitfalls and concerns that may arise in the course of implementation. We argue that reaching consensus on common evaluation benchmarks will foster cumulative progress in psychology, and encourage researchers to place heavier emphasis on the practical utility of scientific models.

Evaluation of Selected Enterprise Reference Models

2011 ◽  
pp. 266-287 ◽  
Author(s):  
Jean-Paul Van Belle
Keyword(s):  
Model Evaluation ◽  
Comprehensive Evaluation ◽  
Evaluation Criteria ◽  
Real Life ◽  
Erp Systems ◽  
Reference Models ◽  
Enterprise Models ◽  
The Many ◽  
Strength Models ◽  
Industrial Strength

This chapter describes a comprehensive evaluation of ten enterprise reference models, including the models underlying the two leading ERP systems (SAP and Baan) and a number of prominent data model libraries. The main purpose of the chapter is to explore how well various model evaluation criteria and the associated metrics can be applied to real-life enterprise models. The analysis is structured into syntactic, semantic and pragmatic criteria. Not all criteria can be measured using clear or unambiguous metrics and some novel, exploratory approaches are suggested. The chapter does not only provide an insight how some of the better-known enterprise models compare against each other, but it also highlights the many practical problems and issues encountered with applying evaluation criteria to industrial-strength models.

Comparison of Selected Model Evaluation Criteria for Maintenance Applications

2004 ◽  
Vol 3 (3) ◽  
pp. 213-224 ◽  
Author(s):  
Ranganath Kothamasu ◽  
J. Shi ◽  
Samuel H. Huang ◽  
H. R. Leep
Keyword(s):  
Model Evaluation ◽  
Evaluation Criteria

Transformation of evaluation criteria of lands withdrawn from agricultural turnover due to radioactive contamination

2020 ◽  
pp. 23-27
Author(s):  
Olga Merzlova
Keyword(s):  
Economic Evaluation ◽  
Radioactive Contamination ◽  
Chernobyl Accident ◽  
Evaluation Criteria ◽  
Contaminated Land ◽  
Degradation Processes ◽  
Agricultural Use ◽  
Land Clearance

One of the measures to eliminate the consequences of the Chernobyl accident was the exclusion of highly contaminated land from agricultural use. Due to the positive dynamics of the radiation situation, the issue of land return becomes relevant. However, in the period of exclusion of these lands the land clearance degradation processes were developing. The second part of the article is devoted to the issue of economic evaluation of the expediency of land return and the mutual coordination of the results of separate stages of complex ecological and economic evaluation. The research was carried out in Mogilev branch Institute of radiology (Republic of Belarus).

Sign in / Sign up

Export Citation Format

Share Document