Program evaluation practices and the training of PhD students in STEM

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Philip M. Reeves ◽  
Jennifer Claydon ◽  
Glen A. Davenport
Keyword(s):  
Program Evaluation ◽  
Graduate Program ◽  
Evidence Based ◽  
Data Sets ◽  
Content Type ◽  
Training Models ◽  
Program Testing ◽  
Stem Programs ◽  
Evaluative Processes ◽  
Faculty And Administrators

Purpose Program evaluation stands as an evidence-based process that would allow institutions to document and improve the quality of graduate programs and determine how to respond to growing calls for aligning training models to economic realities. This paper aims to present the current state of evaluation in research-based doctoral programs in STEM fields. Design/methodology/approach To highlight the recent evaluative processes, the authors restricted the initial literature search to papers published in English between 2008 and 2019. As the authors were motivated by the shift at NIH, this review focuses on STEM programs, though papers on broader evaluation efforts were included as long as STEM-specific results could be identified. In total, 137 papers were included in the final review. Findings Only nine papers presented an evaluation of a full program. Instead, papers focused on evaluating individual components of a graduate program, testing small interventions or examining existing national data sets. The review did not find any documents that focused on the continual monitoring of training quality. Originality/value This review can serve as a resource, encourage transparency and provide motivation for faculty and administrators to gather and use assessment data to improve training models. By understanding how existing evaluations are conducted and implemented, administrators can apply evidence-based methodologies to ensure the highest quality training to best prepare students.

scholarly journals Supporting the initial work of evidence-based improvement cycles through a data-intensive partnership

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Alex J. Bowers ◽  
Andrew E. Krumm
Keyword(s):  
School Leaders ◽  
Data Use ◽  
Evidence Based ◽  
Data Sets ◽  
Complex Data ◽  
Digital Learning ◽  
Data Systems ◽  
Content Type ◽  
Complex Data Sets ◽  
Data Visualizations

Purpose Currently, in the education data use literature, there is a lack of research and examples that consider the early steps of filtering, organizing and visualizing data to inform decision-making. The purpose of this study is to describe how school leaders and researchers visualized and jointly made sense of data from a common learning management system (LMS) used by students across multiple schools and grades in a charter management organization operating in the USA. To make sense of LMS data, researchers and practitioners formed a partnership to organize complex data sets, create data visualizations and engage in joint sensemaking around data visualizations to begin to launch continuous improvement cycles. Design/methodology/approach The authors analyzed LMS data for n = 476 students in Algebra I using hierarchical cluster analysis heatmaps. The authors also engaged in a qualitative case study that examined the ways in which school leaders made sense of the data visualization to inform improvement efforts. Findings The outcome of this study is a framework for informing evidence-based improvement cycles using large, complex data sets. Central to moving through the various steps in the proposed framework are collaborations between researchers and practitioners who each bring expertise that is necessary for organizing, filtering and visualizing data from digital learning environments and administrative data systems. Originality/value The authors propose an integrated cycle of data use in schools that builds on collaborations between researchers and school leaders to inform evidence-based improvement cycles.

Stereotactic radiation treatment planning and follow-up studies involving fused multimodality imaging

2004 ◽  
Vol 101 (Supplement3) ◽  
pp. 326-333 ◽  
Author(s):  
Klaus D. Hamm ◽  
Gunnar Surber ◽  
Michael Schmücking ◽  
Reinhard E. Wurm ◽  
Rene Aschenbach ◽  
...  
Keyword(s):  
Image Fusion ◽  
Treatment Planning ◽  
Radiation Treatment ◽  
Data Sets ◽  
Slice Thickness ◽  
Radiation Treatment Planning ◽  
Content Type ◽  
Follow Up Studies ◽  
Fine Print

Object. Innovative new software solutions may enable image fusion to produce the desired data superposition for precise target definition and follow-up studies in radiosurgery/stereotactic radiotherapy in patients with intracranial lesions. The aim is to integrate the anatomical and functional information completely into the radiation treatment planning and to achieve an exact comparison for follow-up examinations. Special conditions and advantages of BrainLAB's fully automatic image fusion system are evaluated and described for this purpose. Methods. In 458 patients, the radiation treatment planning and some follow-up studies were performed using an automatic image fusion technique involving the use of different imaging modalities. Each fusion was visually checked and corrected as necessary. The computerized tomography (CT) scans for radiation treatment planning (slice thickness 1.25 mm), as well as stereotactic angiography for arteriovenous malformations, were acquired using head fixation with stereotactic arc or, in the case of stereotactic radiotherapy, with a relocatable stereotactic mask. Different magnetic resonance (MR) imaging sequences (T1, T2, and fluid-attenuated inversion-recovery images) and positron emission tomography (PET) scans were obtained without head fixation. Fusion results and the effects on radiation treatment planning and follow-up studies were analyzed. The precision level of the results of the automatic fusion depended primarily on the image quality, especially the slice thickness and the field homogeneity when using MR images, as well as on patient movement during data acquisition. Fully automated image fusion of different MR, CT, and PET studies was performed for each patient. Only in a few cases was it necessary to correct the fusion manually after visual evaluation. These corrections were minor and did not materially affect treatment planning. High-quality fusion of thin slices of a region of interest with a complete head data set could be performed easily. The target volume for radiation treatment planning could be accurately delineated using multimodal information provided by CT, MR, angiography, and PET studies. The fusion of follow-up image data sets yielded results that could be successfully compared and quantitatively evaluated. Conclusions. Depending on the quality of the originally acquired image, automated image fusion can be a very valuable tool, allowing for fast (∼ 1–2 minute) and precise fusion of all relevant data sets. Fused multimodality imaging improves the target volume definition for radiation treatment planning. High-quality follow-up image data sets should be acquired for image fusion to provide exactly comparable slices and volumetric results that will contribute to quality contol.

Mean reversion in corporate leverage: evidence from India

2019 ◽  
Vol 45 (9) ◽  
pp. 1183-1198
Author(s):  
Gaurav S. Chauhan ◽  
Pradip Banerjee
Keyword(s):  
Capital Structure ◽  
Emerging Market ◽  
Simulated Data ◽  
Mean Reversion ◽  
Developed Countries ◽  
Data Sets ◽  
Debt Ratio ◽  
Testing Strategy ◽  
Content Type ◽  
Financing Behavior

Purpose Recent papers on target capital structure show that debt ratio seems to vary widely in space and time, implying that the functional specifications of target debt ratios are of little empirical use. Further, target behavior cannot be adjudged correctly using debt ratios, as they could revert due to mechanical reasons. The purpose of this paper is to develop an alternative testing strategy to test the target capital structure. Design/methodology/approach The authors make use of a major “shock” to the debt ratios as an event and think of a subsequent reversion as a movement toward a mean or target debt ratio. By doing this, the authors no longer need to identify target debt ratios as a function of firm-specific variables or any other rigid functional form. Findings Similar to the broad empirical evidence in developed economies, there is no perceptible and systematic mean reversion by Indian firms. However, unlike developed countries, proportionate usage of debt to finance firms’ marginal financing deficits is extensive; equity is used rather sparingly. Research limitations/implications The trade-off theory could be convincingly refuted at least for the emerging market of India. The paper here stimulated further research on finding reasons for specific financing behavior of emerging market firms. Practical implications The results show that the firms’ financing choices are not only depending on their own firm’s specific variables but also on the financial markets in which they operate. Originality/value This study attempts to assess mean reversion in debt ratios in a unique but reassuring manner. The results are confirmed by extensive calibration of the testing strategy using simulated data sets.

Knowledge needs in the non-profit sector: an evidence-based model of organizational practices

2016 ◽  
Vol 20 (1) ◽  
pp. 23-48 ◽  
Author(s):  
Dinesh Rathi ◽  
Lisa M. Given ◽  
Eric Forcier
Keyword(s):  
Quantitative Data ◽  
Qualitative Data ◽  
Free Text ◽  
Evidence Based ◽  
Theory Approach ◽  
Organizational Practices ◽  
Content Type ◽  
Non Profit ◽  
Types Of Knowledge ◽  
Survey Responses

Purpose – This paper aims to present findings from a study of non-profit organizations (NPOs), including a model of knowledge needs that can be applied by practitioners and scholars to further develop the NPO sector. Design/methodology/approach – A survey was conducted with NPOs operating in Canada and Australia. An analysis of survey responses identified the different types of knowledge essential for each organization. Respondents identified the importance of three pre-determined themes (quantitative data) related to knowledge needs, as well as a fourth option, which was a free text box (qualitative data). The quantitative and qualitative data were analyzed using descriptive statistical analyses and a grounded theory approach, respectively. Findings – Analysis of the quantitative data indicates that NPOs ' needs are comparable in both countries. Analysis of qualitative data identified five major categories and multiple sub-categories representing the types of knowledge needs of NPOs. Major categories are knowledge about management and organizational practices, knowledge about resources, community knowledge, sectoral knowledge and situated knowledge. The paper discusses the results using semantic proximity and presents an emergent, evidence-based knowledge management (KM)-NPO model. Originality/value – The findings contribute to the growing body of literature in the KM domain, and in the understudied research domain related to the knowledge needs and experiences of NPOs. NPOs will find the identified categories and sub-categories useful to undertake KM initiatives within their individual organizations. The study is also unique, as it includes data from two countries, Canada and Australia.

The ingenious marketing of modern paintings

2014 ◽  
Vol 6 (2) ◽  
pp. 211-233
Author(s):  
Thomas M. Bayer ◽  
John Page
Keyword(s):  
Exchange Process ◽  
Art Market ◽  
Late Nineteenth Century ◽  
Data Sets ◽  
Content Type ◽  
Secondary Sources ◽  
Auction Houses ◽  
Art Dealers ◽  
The Media ◽  
Eighteenth And Nineteenth Centuries

Purpose – This paper aims to analyze the evolution of the marketing of paintings and related visual products from its nascent stages in England around 1700 to the development of the modern art market by 1900, with a brief discussion connecting to the present. Design/methodology/approach – Sources consist of a mixture of primary and secondary sources as well as a series of econometric and statistical analyses of specifically constructed and unique data sets that list nearly more than 50,000 different sales of paintings during this period. One set records sales of paintings at various English auction houses during the eighteenth and nineteenth centuries; the second set consists of all purchases and sales of paintings recorded in the stock books of the late nineteenth-century London art dealer, Arthur Tooth, during the years of 1870/1871. The authors interpret the data under a commoditization model first introduced by Igor Kopytoff in 1986 that posits that markets and their participants evolve toward maximizing the efficiency of their exchange process within the prevailing exchange technology. Findings – We found that artists were largely responsible for a series of innovations in the art market that replaced the prevailing direct relationship between artists and patron with a modern market for which painters produced works on speculation to be sold by enterprising middlemen to an anonymous public. In this process, artists displayed a remarkable creativity and a seemingly instinctive understanding of the principles of competitive marketing that should dispel the erroneous but persistent notion that artistic genius and business savvy are incompatible. Research limitations/implications – A similar marketing analysis could be done of the development of the art markets of other leading countries, such as France, Italy and Holland, as well as the current developments of the art market. Practical implications – The same process of the development of the art market in England is now occurring in Latin America and China. Also, the commoditization process continues in the present, now using the Internet and worldwide art dealers. Originality/value – This is the first article to trace the historical development of the marketing of art in all of its components: artists, dealers, artist organizations, museums, curators, art critics, the media and art historians.

Improving recommender systems’ performance on cold-start users and controversial items by a new similarity model

2016 ◽  
Vol 12 (2) ◽  
pp. 126-149 ◽  
Author(s):  
Masoud Mansoury ◽  
Mehdi Shajari
Keyword(s):  
Real World ◽  
Design Methodology ◽  
Cold Start ◽  
Selection Function ◽  
Data Sets ◽  
Real World Data ◽  
Content Type ◽  
User Similarity ◽  
Active User ◽  
Similarity Model

Purpose This paper aims to improve the recommendations performance for cold-start users and controversial items. Collaborative filtering (CF) generates recommendations on the basis of similarity between users. It uses the opinions of similar users to generate the recommendation for an active user. As a similarity model or a neighbor selection function is the key element for effectiveness of CF, many variations of CF are proposed. However, these methods are not very effective, especially for users who provide few ratings (i.e. cold-start users). Design/methodology/approach A new user similarity model is proposed that focuses on improving recommendations performance for cold-start users and controversial items. To show the validity of the authors’ similarity model, they conducted some experiments and showed the effectiveness of this model in calculating similarity values between users even when only few ratings are available. In addition, the authors applied their user similarity model to a recommender system and analyzed its results. Findings Experiments on two real-world data sets are implemented and compared with some other CF techniques. The results show that the authors’ approach outperforms previous CF techniques in coverage metric while preserves accuracy for cold-start users and controversial items. Originality/value In the proposed approach, the conditions in which CF is unable to generate accurate recommendations are addressed. These conditions affect CF performance adversely, especially in the cold-start users’ condition. The authors show that their similarity model overcomes CF weaknesses effectively and improve its performance even in the cold users’ condition.

Public administration approach

2014 ◽  
Vol 35 (6/7) ◽  
pp. 495-507 ◽  
Author(s):  
Jurgita Rudžionienė ◽  
Jaroslav Dvorak
Keyword(s):  
Performance Evaluation ◽  
Public Administration ◽  
Information Services ◽  
Evidence Based ◽  
Economic Aspects ◽  
Content Type ◽  
Evaluation Functions ◽  
Services Evaluation ◽  
Library Evaluation ◽  
Evaluation Measurement

Purpose – The purpose of this paper is to define the problem and to initiate discussion on library evaluation as significant part of institutional evidence-based management from public administration approach. Design/methodology/approach – In order to fulfilling the purpose, special attention to present the concepts of valuing information, library performance evaluation, measurement, etc. is drawn, main evaluation functions are analysed. Economic aspects of information services vs intellectual ones are discussed. Consistent patterns and principles of public administration as well as possibilities of public administration influence in creation of systematic base of library performance evaluation as well as of information services impact to the user are analysed. Findings – The paper provides insights about different aspects of information services evaluation. Results of analysis of economic aspects of information services vs intellectual ones are presented, consistent patterns and principles of public administration, possibilities of public administration influence in creation of systematic base of library performance evaluation as well as of information services impact to the user possibilities are presented. Originality/value – The paper fulfills need to study how public administration could involve library evaluation as tool for evidence-based decision making.

Building holistic and agile collection development and assessment

2015 ◽  
Vol 16 (1) ◽  
pp. 62-85 ◽  
Author(s):  
Cheri Jeanette Duncan ◽  
Genya Morgan O'Gara
Keyword(s):  
Quantitative Data ◽  
Holistic Approach ◽  
Assessment Model ◽  
Data Sets ◽  
Collection Development ◽  
Content Type ◽  
Qualitative And Quantitative ◽  
Institutional Goals ◽  
Scholarly Output ◽  
Use Of Data

Purpose – The purpose of this paper is to examine the development of a flexible collections assessment rubric comprised of a suite of tools for more consistently and effectively evaluating and expressing a holistic value of library collections to a variety of constituents, from administrators to faculty and students, with particular emphasis to the use of data already being collected at libraries to “take the temperature” of how responsive collections are in supporting institutional goals. Design/methodology/approach – Using a literature review, internal and external conversations, several collections pilot projects, and a variety of other investigative mechanisms, this paper explores methods for creating a more flexible, holistic collection development and assessment model using both qualitative and quantitative data. Findings – The products of scholarship that academic libraries include in their collections are expanding exponentially and range from journals and monographs in all formats, to databases, data sets, digital text and images, streaming media, visualizations and animations. Content is also being shared in new ways and on a variety of platforms. Yet the framework for evaluating this new landscape of scholarly output is in its infancy. So, how do libraries develop and assess collections in a consistent, holistic, yet agile, manner? Libraries must employ a variety of mechanisms to ensure this goal, while remaining flexible in adapting to the shifting collections environment. Originality/value – In so much as the authors are aware, this is the first paper to examine an agile, holistic approach to collections using both qualitative and quantitative data.

scholarly journals Enhanced Care Assistant Training to Address the Workforce Crisis in Home Care: Changes Related to Job Satisfaction and Career Commitment

2007 ◽  
Vol 8 (2) ◽  
pp. 71-81 ◽  
Author(s):  
Constance L. Coogle ◽  
Iris A. Parham ◽  
Rita Jablonski ◽  
Jason A. Rachel
Keyword(s):  
Job Satisfaction ◽  
Home Care ◽  
Career Commitment ◽  
Management Training ◽  
Data Sets ◽  
Term Care ◽  
Training Models ◽  
Intrinsic Job Satisfaction

Changes in job satisfaction and career commitment were observed as a consequence of a geriatric case management training program focusing on skills development among personal care attendants in home care. A comparison of pretraining and posttraining scores uncovered a statistically significant increase in Intrinsic Job Satisfaction scores for participants 18–39 years of age, whereas levels declined among the group of middle aged participants and no change was observed among participants age 52 and older. On the other hand, a statistically significant decline in Extrinsic Job Satisfaction was documented over all participants, but this was found to be primarily due to declines among participants 40–51 years of age. When contacted 6–12 months after the training series had concluded, participants indicated that the training substantially increased the likelihood that they would stay in their current jobs and improved their job satisfaction to some extent. A comparison of pretraining and posttraining scores among participants providing follow-up data revealed a statistically significant improvement in levels of Career Resilience. These results are discussed as they relate to similar training models and national data sets, and recommendations are offered for targeting future educational programs designed to address the long-term care workforce shortage.

A systematic review of machine learning-based missing value imputation techniques

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Tressy Thomas ◽  
Enayat Rajabi
Keyword(s):  
Machine Learning ◽  
Selection Process ◽  
Evaluation Metrics ◽  
Correct Prediction ◽  
Data Sets ◽  
Data Set ◽  
Missing Value ◽  
Content Type ◽  
Missing Value Imputation ◽  
Literature Reviews

PurposeThe primary aim of this study is to review the studies from different dimensions including type of methods, experimentation setup and evaluation metrics used in the novel approaches proposed for data imputation, particularly in the machine learning (ML) area. This ultimately provides an understanding about how well the proposed framework is evaluated and what type and ratio of missingness are addressed in the proposals. The review questions in this study are (1) what are the ML-based imputation methods studied and proposed during 2010–2020? (2) How the experimentation setup, characteristics of data sets and missingness are employed in these studies? (3) What metrics were used for the evaluation of imputation method?Design/methodology/approachThe review process went through the standard identification, screening and selection process. The initial search on electronic databases for missing value imputation (MVI) based on ML algorithms returned a large number of papers totaling at 2,883. Most of the papers at this stage were not exactly an MVI technique relevant to this study. The literature reviews are first scanned in the title for relevancy, and 306 literature reviews were identified as appropriate. Upon reviewing the abstract text, 151 literature reviews that are not eligible for this study are dropped. This resulted in 155 research papers suitable for full-text review. From this, 117 papers are used in assessment of the review questions.FindingsThis study shows that clustering- and instance-based algorithms are the most proposed MVI methods. Percentage of correct prediction (PCP) and root mean square error (RMSE) are most used evaluation metrics in these studies. For experimentation, majority of the studies sourced the data sets from publicly available data set repositories. A common approach is that the complete data set is set as baseline to evaluate the effectiveness of imputation on the test data sets with artificially induced missingness. The data set size and missingness ratio varied across the experimentations, while missing datatype and mechanism are pertaining to the capability of imputation. Computational expense is a concern, and experimentation using large data sets appears to be a challenge.Originality/valueIt is understood from the review that there is no single universal solution to missing data problem. Variants of ML approaches work well with the missingness based on the characteristics of the data set. Most of the methods reviewed lack generalization with regard to applicability. Another concern related to applicability is the complexity of the formulation and implementation of the algorithm. Imputations based on k-nearest neighbors (kNN) and clustering algorithms which are simple and easy to implement make it popular across various domains.

Sign in / Sign up

Export Citation Format

Share Document