Evaluation in the Australian Public Service: Formerly practised – Not yet embedded

This article examines what impeded programme evaluation from being embedded in the Australian Public Service (APS), being relevant to the Australian Government’s current priority of embedding evaluation in the APS. It draws on a case study of evaluation as the major element of the 1980s APS ‘Managing for Results’ (MfR) reform and the reasons for evaluation’s later demise. During MfR, evaluation was intended to demonstrate the effectiveness of APS programmes. Although evaluation was incorporated into APS practice by 1992, after 1997, evaluation was no longer required. Currently, agencies must demonstrate their annual non-financial performance over 4 years under the Public Governance, Performance and Accountability Act 2013, with evaluation being recommended to support this requirement. It is pertinent to current Government consideration of a National Indigenous Evaluation Strategy, which supports the creation of an independent Evaluator-General to embed APS evaluation practice.

Neuroticism as an underlying construct in youth emotional disorders

Structural research on neuroticism, as indicated by Big Five personality traits and/or internalizing symptoms, has been conducted with youth. However, no structural research has investigated neuroticism as characterized by transdiagnostic risk factors such as distress tolerance (DT), negative affect (NA), and avoidance. No study has investigated whether DT, NA, and avoidance, as a group, are associated with anxiety, depressive, obsessive-compulsive (OC) symptoms, and independent evaluator (lE)-rated symptom severity in a clinical sample of youth. The purpose of the current investigation was to understand the proportion of variance in anxiety, depressive, OC symptoms, and independent evaluator-rated global symptom severity by a latent construct of neuroticism, as indicated by these modifiable features in youth with emotional disorders among a sample of 121 adolescents (ages 13–18, 51.2% female). A latent neuroticism factor was significantly associated with greater youth- and parent-reported anxiety, depressive, and OC symptoms, and greater IE-rated global severity.

Book Review: Encyclopedia of Public Health: Principles, Peoples, and Programs

The Encyclopedia of Public Health: Principles, Peoples, and Programs, a new addition to Greenwood’s health reference catalog, provides a solid resource for libraries looking for a good, low-cost encyclopedia for their public health collection. Dr. Sally Kuykendall, the editor, is a professor of health services at St. Joseph’s University in Philadelphia and an independent evaluator of public health programs. She has assembled a knowledgeable team of public health experts, health scientists, and medical historians as contributors, who present topics relating to public health in an easily-readable format for general readers. The two volumes present signed articles alphabetically by entry, with cross-references and bibliographies. The text is scattered throughout with black and white illustrations. Both volumes contain a contents section, a guide to related topics that groups articles by theme, a chronology, and an introduction. Volume 2 contains a glossary, a list of organizations and contributors, and a comprehensive index, as well as a section on “Controversies in public health,” which discusses hot button issues like anti-vaccination trends and health care for undocumented immigrants. Kuykendall maintains an even tone and fair treatment of all sides, allowing both pro- and anti-sides of an issue their say.

LIFE QUALITY EVALUATION USING “TIME TRADE OFF” METHOD FOR RHEUMATOID HANDS

ABSTRACT Objective: Rheumatoid arthritis is a prevalent disease in the population (range 0.5% to 1%) and involves both orthopedic and rheumatologic treatment. The Time Trade-Off (TTO) technique, which determines the number of years the patient or the professional would be allowed before a successful procedure in terms of life expectancy and value of the procedure, has been gaining ground in clinical protocols. From this standpoint, we sought to compare evaluations provided by the patients, orthopedists, and rheumatologists in determining the TTO and to correlate their responses with the clinical repercussions using previously established scores such as the Brief Michigan Hand Questionnaire and the Disease Activity Score-28 (DAS-28). Methods: A prospective study was conducted that involved 37 patients with rheumatoid arthritis, orthopedists, and rheumatologists. The TTO questionnaire was administered by an independent evaluator for evaluation using the DAS-28 and the Brief Michigan Hand Questionnaire. Results: The descriptive analysis revealed similar medians between the orthopedists, rheumatologists, and patients for single assessments. However, there was a weak correlation between the results from the patient and rheumatologist, the patient and Brief Michigan Questionnaire, and those of the orthopedic surgeon and the DAS-28. Conclusion: Similar median values demonstrated equivalent TTO among the orthopedist, rheumatologist, and patient. However, given the weak correlations between the scores, it was not possible to substitute results using a single evaluation scale. Level of Evidence II, Prognostic Studies.

Preoperative Surgical Discussion and Information Retention by Patients

Background: To assess how much information communicated to patients is understood and retained after preoperative discussion of upper extremity procedures. Methods: A prospective study was designed by recruiting patients prior to undergoing upper extremity surgical procedures after a detailed discussion of their operative technique, postoperative care and treatment outcomes. Patients were given the same 20-item questionnaire to fill out twice, at two pre operative visits. An independent evaluator filled out a third questionnaire as a control. Various discussion points of the survey were compared among the 3 questionnaires and retained information and perceived comprehension were evaluated. Results: The average patients’ age was 50.3 (27–75) years The average time between the two surveys preoperative 1 and preoperative 2 was 40.7 (7–75) days,. The average patient had approximately 2 years of college or an associate’s degree. Patients initially retained 73% (52–90%) of discussion points presented during preoperative 1 and 61% (36–85%) of the information at preoperative 2 p = .002. 50% of patients felt they understood 100% of the discussion, this dropped to only 10% at their preoperative 2 visit. 15% of our patients did not know what type of anesthesia they were having at preoperative 2. Conclusions: A communication barrier between patients and physicians exists when patients are informed about their preoperative surgical discussion. The retention of information presented is worsened with elapsing time from the initial preoperative discussion to the second preoperative visit immediately prior to surgery. Methods to enhance patients’ retention of information prior to surgery must be sought and implemented which will improve patients’ treatment outcome.

Using Videoconferencing for Elementary/Secondary Teacher Professional Development

Working within a large geographic area, McDonald Observatory lacks the resources to send staff to present professional development workshops to teachers – and only limited numbers of teachers have the resources to attend workshops at our observatory. Our solution is the development of a new program to bring the workshop to teachers in their own communities through videoconferencing. Each workshop location has a co-facilitator who prepared for his/her duties through an orientation/training session held at McDonald Observatory. At the observatory, they experienced a variety of activities and selected the ones most suitable for the grade-levels of the teachers in their region; they recruit the teachers for the local workshop. Each videoconference session includes pre/post assessment of the participants, an interactive videoconference with an expert presenter, and assistance from the co-facilitator who manages the materials and assists with the activities. Through use of this technology, we expect to reach 500 teachers. An independent evaluator is preparing formative and summative evaluation for the project.

Phase 1 of Integrated EMDR An Abortive Treatment for Migraine Headaches

Forty-three individuals diagnosed with classic or common migraine headache were randomly assigned to either phase 1 of integrated eye movement desensitization reprocessing (EMDR) treatment or a standard care medication treatment. Integrated EMDR combines diaphragmatic breathing, cranial compression, and EMDR for abortive migraine treatment. The comparison standard care medication group received various abortive medications, including Demerol, DHE, oral triptans, Excedrin, Fiorinal, Percocet, Toradol, and Vicodin. Participants were treated during mid- to late-stage acute migraine and assessed by an independent evaluator at pretreatment, posttreatment, 24 hours, 48 hours, and 7 days for migraine pain level. Both standard care medication and integrated EMDR treatment groups demonstrated reduced migraine pain levels immediately at posttreatment, 24 hours, 48 hours, and 7 days. However, integrated EMDR treatment reduced or eliminated migraine pain with greater rapidity and showed significantly greater improvement compared to standard care medication immediately posttreatment.

Inter-rater reliability and factor analysis of the Brazilian version of the Schedule for the Assessment of Insight: Expanded Version (SAI-E)

OBJECTIVES: The Schedule for the Assessment of Insight - Expanded Version consists of 11 items that encompass: awareness of having a mental illness, ability to rename psychotic phenomena as abnormal, and compliance with treatment. The objective of the study was to evaluate the inter-rater reliability and to study the factorial structure of the Brazilian version of the instrument. METHOD: The Brazilian version of the Schedule for the Assessment of Insight - Expanded Version was used for the assessment of insight of 109 psychotic inpatients, 60 of whom had the interview tape-recorded in order to be scored by an independent evaluator. Intraclass correlation coefficient (ICC) was adopted as the inter-rater reliability coefficient. In the factor analysis, principal components analysis and Varimax rotation were adopted. RESULTS: Inter-rater reliability coefficients from good to excellent were found for the individual items of the Schedule for the Assessment of Insight - Expanded Version with ICC values ranging from 0.54 to 0.82. Regarding the total score, inter-rater reliability was excellent, with ICC = 0.90. A factorial structure similar to the one obtained by the original version of the Schedule for the Assessment of Insight - Expanded Version was found, with 3 factors accounting for 71.72% of variance. CONCLUSION: In the Brazilian context, the Schedule for the Assessment of Insight - Expanded Version presented good inter-rater reliability and factorial structure compatible to the insight dimensions that are intended to be evaluated.

Conventional cautions regarding online information: Do they predict accuracy of breast cancer websites?

6086 Introduction: Cancer patients and oncologists are increasingly using the Internet. Online information is affecting patient decisions regarding treatment. Therefore, multiple organizations published advice on how to screen out problematic information online. In the present study, we sought to determine whether existing quality assessment tools can screen out false or inaccurate breast cancer information online. Methods: Using 15 representative search queries on five popular search engines and only the first page of results, we generated a list of 1,585 Web pages. After eliminating irrelevant, broken and duplicate links 343 unique web pages were downloaded and evaluated for website characteristics, information accuracy and 15 quality criteria: authorship identified, author credentials identified, physician credentials stated, author affiliation stated, sources clear, general disclosures, attribution/references provided, disclosure of ownership, date created displayed, date of last update displayed, date of creation or last updated displayed, editorial process stated, internal search engine present, feedback mechanism provided and copyright notice. Quality criteria were evaluated by an independent evaluator blinded to the accuracy information. Accuracy was determined independently by a pair of clinically trained evaluators. Results: Of 343 pages evaluated, 41 false/misleading statements were displayed on 18 distinct pages. The presence of an internal search engine was correlated most strongly (r = .196, r2 = .038) with health information accuracy. However, all quality criteria (alone or in combinations tested exhaustively) were linearly associated with less than 7% of the variance in number of inaccuracies per website. Conclusion: Currently available technical quality criteria do not identify false or misleading online information about breast cancer. At this time, directing patients to websites with known content appears to be the only way to ensure patients will not encounter false or misleading online information. No significant financial relationships to disclose.