Evaluation Strategies for Cybersecurity Training Methods: A Literature Review

AbstractDespite advances in the field of missense variant effect prediction, the real clinical utility of current computational approaches remains rather limited. There is a large difference in performance metrics reported by developers and those observed in the real world. Most currently available predictors suffer from one or more types of circularity in their training and evaluation strategies that lead to overestimation of predictive performance. We present a generic strategy that is independent of dataset properties and algorithms used, to deal with circularity in the training phase. This results in more robust predictors and evaluation scores that accurately reflect the real-world performance of predictive models. Additionally, we show that commonly used training methods can have an adverse impact on model performance and lead to gross overestimation of true predictive performance.

Download Full-text

“Advocates Change the World; Evaluation Can Help”: A Literature Review and Key Insights from the Practice of Advocacy Evaluation

Canadian Journal of Program Evaluation ◽

10.3138/cjpe.31039 ◽

2017 ◽

Vol 32 (1) ◽

Cited By ~ 3

Author(s):

Juniper Glass

Keyword(s):

Literature Review ◽

Systems Change ◽

The United States ◽

Change Initiatives ◽

Priority List ◽

Non Profit ◽

Evaluation Practice ◽

Evaluation Strategies ◽

Rational Evaluation ◽

Considerable Resource

It is only since the new millennium that assessments of policy and systems change initiatives have been given much attention in practice or in research. A comprehensive literature review of advocacy in human services non-profit organizations (NGOs) has found a “lack of systemic, rational evaluation and measurement of the effectiveness of advocacy.” Similarly, in a survey of 211 NGOs that undertake advocacy, only one in four (24.6 percent) reported that this work had been evaluated. While the survey was conducted in the United States, it is likely that Canadian NGOs are in a similar situation, not because of a lack of interest or value for evaluation but, rather, because assessing systems change initiatives is challenging territory and NGOs face considerable resource and time constraints that put such evaluation low on the priority list. This article provides a review of key insights from advocacy evaluation practice and research that may help orient and inform NGOs as they decide on evaluation strategies. I will outline the state of the field of systems - and policy -change evaluation in North America as well as its benefits and challenges. Finally, a synthesis of the main steps in advocacy evaluation planning and implementation offers a broad map to NGOs seeking to enhance this practice in their own organizations.Ce n’est que depuis le dernier millénaire que l’on s’intéresse davantage, en pratique ou en recherche, aux initiatives visant à créer des changements dans les systèmes et les politiques. Une analyse documentaire exhaustive des activités de plaidoyer dans les organisations à but non lucratif (ONG – organisations non gouvernementales) offrant des services humanitaires montre « un manque d’évaluation et de mesure systémique en ce qui a trait à l’efficacité du plaidoyer ». De même, dans un sondage de 211 ONG ayant des activités de plaidoyer, seule une sur quatre (24,6 pourcent) ont indiqué que ce travail avait été évalué. Le sondage a été effectué aux États-Unis, mais il est probable que les ONG canadiennes se trouvent dans une situation semblable, non par manque d’intérêt ou de reconnaissance de la valeur de l’évaluation, mais plutôt parce qu’il n’est pas facile d’évaluer des changements de nature systémique et que les ONG doivent composer avec des contraintes de temps et de ressources qui font que l’évaluation n’est pas une priorité. Cet article propose des éléments clés tirés de la pratique et de la recherche de type « advocacy » qui permettront d’informer et de guider les ONG dans leur choix de stratégies d’évaluation. Je ferai état des avancées dans le champ de l’analyse des politiques et des systèmes en Amérique du Nord, de même des avantages et des défi s qui y sont liés. Finalement, une synthèse des principales étapes de la planification et de la réalisation d’une évaluation de type « advocacy » pourra guider les ONG qui souhaiteraient améliorer cette pratique dans leur propre organisation.

Download Full-text

Individual competences for sustainable purchasing and supply management (SPSM)

International Journal of Physical Distribution & Logistics Management ◽

10.1108/ijpdlm-01-2018-0036 ◽

2019 ◽

Vol 49 (3) ◽

pp. 287-304 ◽

Cited By ~ 6

Author(s):

Heike Schulze ◽

Lydia Bals ◽

Thomas E. Johnsen

Keyword(s):

Literature Review ◽

Professional Training ◽

Explicit Knowledge ◽

Supply Management ◽

Organizational Level ◽

Training Methods ◽

Supply Networks ◽

Content Type ◽

Research Gap ◽

The Individual

PurposeImplementing sustainability into global supply networks remains a challenge for companies. Purchasing and supply management (PSM) interacts closely with supply network actors, thus influencing how the firm’s value creation is delivered. While previous sustainable PSM (SPSM) research has shed light on how to manage sustainability on an organizational level, the individual competences PSM professionals require are less understood. The paper aims to discuss this issue.Design/methodology/approachThe authors conducted a systematic literature review to determine the current research coverage of specific competences and knowledge required to implement sustainability. The authors complemented this with data from 46 interviews with practitioners. From coding the data with NVivo, a first comprehensive competence overview for SPSM was developed.FindingsThe literature review results, complemented with interview data, highlight that functional-oriented, cognition-oriented, social-oriented and meta-oriented competences form part of a comprehensive SPSM competence model. We propose a framework that includes these, and integrates two behavioral moderators on the organizational level, i.e. situational enabling, as well as empowerment and obligation.Research limitations/implicationsWhile the proposed framework provides a basic first systematization of SPSM competences, further research is needed to extend it. There is ample opportunity to shed further light on both individual and organizational-level factors that influence the application of SPSM competences, and therefore SPSM behavior.Practical implicationsThe results have implications for higher education and professional training programs in companies. The framework provides an overview of competences needed for SPSM. The discussion highlights the need to apply education and training methods for different types of competences that are suitable for conveying implicit knowledge apart from explicit knowledge.Originality/valueAdressing a current research gap in sustainability-related competences in PSM, the overall framework highlights SPSM competences of interest to both scholars and managers alike.

Download Full-text

Formación de Animadores de Deporte para Todos (Training of coordinators of Sports for Everyone)

Retos ◽

10.47197/retos.v0i17.34671 ◽

2015 ◽

pp. 62-66

Author(s):

José Antonio Rebollo González ◽

Pedro Sáenz-López Buñuel

Keyword(s):

Literature Review ◽

Key Words ◽

Current Situation ◽

Evaluation Strategies ◽

Curricular Elements

Con los datos obtenidos en el Diagnostico de la situación actual de la formación de Animadores de Deporte para Todos, con la base de la estructura de los cursos que se realizaron en nuestra Provincia y con la revisión bibliográfica que se ha llevado a cabo, se ha diseñado un curso de Animador de Deporte para Todos (Nivel 1). Se han llevado a cabo modificaciones en todos los elementos curriculares, intentando en todo momento adaptar la formación de estos Animadores a la situación actual y tratando de equiparar la estructura, a la formación de cualquier técnico deportivo. En este sentido, hemos aumentado los contenidos que se imparten en la actualidad, dividiéndolos en un bloque común y otro específico. Los objetivos se han formulado con arreglo al nuevo diseño. Se ha puesto en marcha una propuesta metodológica consensuada con el profesorado, con un esquema tanto para las sesiones teóricas como para las prácticas. Se ha tratado de unificar las estrategias de evaluación intentando que hubiera coherencia con la propuesta metodológica. La distribución horaria se ha hecho con arreglo a lo que marca la Orden 3310/2002, que regula la formación en materia deportiva.Palabra clave: Deporte para todos, Recreación y formación.Abstract: With the data obtained in the diagnosis of the current situation of training «Sport for All», regarding the base of the structure of the courses that were conducted in our Province and after the literature review that was conducted, we have created a new course of coordinators of Sports for Everyone. There have been modifications in all curricular elements, always trying to adapt the training of animators to the current situation and trying to align the structure to the training of any sport’s professional. In this regard, we have increased the content being taught in the present, dividing them into a common and a specific block. The objectives have been formulated under the new design. It has been put in place a methodology agreed with the teachers, with a schedule for meetings as for the theoretical and as well as for the practical subjects. We have tried to unify the evaluation strategies to fit consistency with the proposed methodology. The planning was done according to the Order 3310/2002, which regulates the training of sports. Key words: Sports for Everyone, Recreation, Formation.

Download Full-text

Reinforcement learning for personalization: A systematic literature review

Data Science ◽

10.3233/ds-200028 ◽

2020 ◽

Vol 3 (2) ◽

pp. 107-147 ◽

Cited By ~ 1

Author(s):

Floris den Hengst ◽

Eoin Martino Grua ◽

Ali el Hassouni ◽

Mark Hoogendoorn

Keyword(s):

Reinforcement Learning ◽

Literature Review ◽

Systematic Literature Review ◽

Digital Systems ◽

Continuous Control ◽

Game Playing ◽

Realistic Evaluation ◽

Evaluation Strategies ◽

Major Application ◽

Future Work

The major application areas of reinforcement learning (RL) have traditionally been game playing and continuous control. In recent years, however, RL has been increasingly applied in systems that interact with humans. RL can personalize digital systems to make them more relevant to individual users. Challenges in personalization settings may be different from challenges found in traditional application areas of RL. An overview of work that uses RL for personalization, however, is lacking. In this work, we introduce a framework of personalization settings and use it in a systematic literature review. Besides setting, we review solutions and evaluation strategies. Results show that RL has been increasingly applied to personalization problems and realistic evaluations have become more prevalent. RL has become sufficiently robust to apply in contexts that involve humans and the field as a whole is growing. However, it seems not to be maturing: the ratios of studies that include a comparison or a realistic evaluation are not showing upward trends and the vast majority of algorithms are used only once. This review can be used to find related work across domains, provides insights into the state of the field and identifies opportunities for future work.

Download Full-text

MODERN LANGUAGE TRAINING METHODS AND THEIR APPLICATION IN CZECH AND SLOVAK BUSINESSES: A LITERATURE REVIEW

CBU International Conference Proceedings ◽

10.12955/cbup.v4.777 ◽

2016 ◽

Vol 4 ◽

pp. 336-339

Author(s):

Adam Dékány

Keyword(s):

Czech Republic ◽

Literature Review ◽

Teaching Methods ◽

English Proficiency ◽

Language Teaching ◽

Current Level ◽

Training Methods ◽

Language Training ◽

The Czech Republic ◽

Modern Language

This literature review outlines modern language-teaching methods used in the Czech Republic and Slovakia. It attempts to identify the current level of English proficiency in Slovakia and the Czech Republic. In addition, it aims to summarize modern methods of language teaching that became available with advent of information technologies. The review has three major parts. The first part examines the current level of English proficiency in Slovakia and the Czech Republic. The second part describes the commonly recognized language proficiency levels established by the Council of Europe. The last part of the review introduces some of the most recent language-training methods that have become widely available due to the development of IT in the last few years. The conclusion includes recommendations about the use of IT for language training. This paper provides an overview of the subject, focusing mainly on readers who are either new to the topic or looking for a simple overview of the present situation. Based on the findings, it can be concluded that the Czech Republic and Slovakia have an average level of English proficiency and at least nine modern teaching methods can be used to improve an individual’s language skills.

Download Full-text

Do aversive-based training methods actually compromise dog welfare?: A literature review

Applied Animal Behaviour Science ◽

10.1016/j.applanim.2017.07.001 ◽

2017 ◽

Vol 196 ◽

pp. 1-12 ◽

Cited By ~ 16

Author(s):

Joana Guilherme Fernandes ◽

I. Anna S. Olsson ◽

Ana Catarina Vieira de Castro

Keyword(s):

Literature Review ◽

Training Methods

Download Full-text

Effectiveness of Hospital Staff Mass-Casualty Incident Training Methods: A Systematic Literature Review

Prehospital and Disaster Medicine ◽

10.1017/s1049023x00001771 ◽

2004 ◽

Vol 19 (3) ◽

pp. 191-199 ◽

Cited By ~ 100

Author(s):

Edbert B. Hsu ◽

Mollie W. Jenckes ◽

Christina L. Catlett ◽

Karen A. Robinson ◽

Carolyn Feuerstein ◽

...

Keyword(s):

Literature Review ◽

Hospital Staff ◽

Mass Casualty Incident ◽

Evaluation Methods ◽

Mass Casualty ◽

Lessons Learned ◽

Current Evidence ◽

Training Methods ◽

Incident Command ◽

Disaster Training

AbstractIntroduction:Recently, mass-casualty incident (MCI) preparedness and training has received increasing attention at the hospital level.Objectives:To review the existing evidence on the effectiveness of disaster drills, technology-based interventions and tabletop exercises in training hospital staff to respond to an MCI.Methods:A systematic, evidence-based process was conducted incorporating expert panel input and a literature review with the key terms: “mass casualty”, “disaster”, “disaster planning”, and “drill”. Paired investigators reviewed citation abstracts to identify articles that included evaluation of disaster training for hospital staff. Data were abstracted from the studies (e.g., MCI type, training intervention, staff targeted, objectives, evaluation methods, and results). Study quality was reviewed using standardized criteria.Results:Of 243 potentially relevant citations, twenty-one met the defined criteria. Studies varied in terms of targeted staff, learning objectives, outcomes, and evaluation methods. Most were characterized by significant limitations in design and evaluation methods. Seventeen addressed the effectiveness of disaster drills in training hospital staff in responding to an MCI, four addressed technology-based interventions, and none addressed tabletop exercises. The existing evidence suggests that hospital disaster drills are effective in allowing hospital employees to become familiar with disaster procedures, identify problems in different components of response (e.g., incident command, communications, triage, patient flow, materials and resources, and security) and provide the opportunity to apply lessons learned to disaster response. The strength of evidence on other training methods is insufficient to draw valid recommendations.Conclusions:Current evidence on the effectiveness of MCI training for hospital staff is limited. A number of studies suggest that disaster drills can be effective in training hospital staff. However, more attention should be directed to evaluating the effectiveness of disaster training activities in a scientifically rigorous manner.

Download Full-text

Management and rehabilitation of patients with persistent olfactory disorders (hypo- and anosmia) (literature review)

Neuromuscular Diseases ◽

10.17650/2222-8721-2021-11-2-12-16 ◽

2021 ◽

Vol 11 (2) ◽

pp. 12-16

Author(s):

V. B. Voitenkov ◽

E. V. Ekusheva ◽

G. V. Lavrenova ◽

N. V. Skripchenko ◽

M. A. Bedova ◽

...

Keyword(s):

Electrical Stimulation ◽

Literature Review ◽

Olfactory Function ◽

Rational Approach ◽

Training Methods ◽

Review Of The Literature ◽

Ethmoid Bone ◽

Common View ◽

Olfactory Training ◽

Stimulation Of

We present a review of the literature and our own data on the management and rehabilitation of impairment or persis‑tent loss of the olfactory function – hypo‑ and anosmia. Approaches for restoring impaired olfactory function can be divided into olfactory training, pharmacological and physiotherapy. Smell training is carried out according to a number of protocols that differ mainly in the used arsenal of smells, as well as in the frequency of their presentation. Pharma‑cologically, it is proposed to use steroids used both topically (intranasally) and systemically; there is no common view on the effectiveness of this approach. Physiotherapy involve electrical stimulation of the olfactory filaments in the area of their exit into the nasal cavity (lateral masses of the ethmoid bone). In our opinion the most rational approach is the use of training methods using different smells.

Download Full-text

Toward Establishing Entrepreneurship Education and Training Programmes in a Multinational Arab University

Journal of Education and Training Studies ◽

10.11114/jets.v7i1.3833 ◽

2018 ◽

Vol 7 (1) ◽

pp. 1

Author(s):

Abbas Abdelkarim

Keyword(s):

Literature Review ◽

Entrepreneurship Education ◽

Entrepreneurial Intention ◽

Training Methods ◽

Target Groups ◽

Entrepreneurship Training ◽

Survey Results ◽

Training Programmes ◽

High Level ◽

And Training

This paper rests on a survey among students of Arab Open University (AOU) that covered 6,369 students from all branches (in eight countries) and across all four colleges. It endeavours to raise a case for introducing entrepreneurship education in AOU. The Survey results show a surprisingly high level of entrepreneurial intention among the students, and the overwhelming majority of them are demanding introduction of entrepreneurship education. The Paper uses results of the Survey to present how the students of AOU desire their entrepreneurship education and entrepreneurship training programmes to be organised, and to identify the target groups of each of the two programmes. Based on these results and on selected literature review of the concepts of entrepreneurship, entrepreneurship education and entrepreneurship training, an outline of entrepreneurship curricula and of teaching and training methods are suggested. Both curricula and methods advanced could be of relevance beyond the specific case of AOU.

Download Full-text