The Emergence of an Item-writing Technology

1980 ◽  
Vol 50 (2) ◽  
pp. 293-314 ◽  
Author(s):  
Gale Roid ◽  
Tom Haladyna
Keyword(s):  
Empirical Studies ◽  
Achievement Tests ◽  
Emerging Technology ◽  
Achievement Testing ◽  
Design Domain ◽  
Item Development ◽  
Item Writing ◽  
Writing Technology ◽  
Facet Design

The emerging technology of item writing for achievement tests is reviewed. Several different approaches to item development are discussed. A continuum of item-writing methods is proposed ranging from informal-subjective methods to algorithmic-objective methods. Examples of techniques include objective-based item writing, amplified objectives, item forms, facet design, domain-referenced concept testing, and computerized techniques. Each item-writing technique is critically reviewed, and empirical studies of methods are described. Recommendations for further research and for applications to achievement testing are presented.

scholarly journals Relating a concordance-based cloze test to the model of communicative language ability: a verbal protocol study

2022 ◽  
Vol 12 (1) ◽  
Author(s):  
Kunlaphak Kongsuwannakul
Keyword(s):  
Native Speakers ◽  
Empirical Studies ◽  
First Language ◽  
Item Type ◽  
Language Ability ◽  
Verbal Reports ◽  
Test Taking ◽  
Item Writing ◽  
Protocol Study ◽  
Cloze Test

AbstractThe literature of empirical studies on the concordance-based cloze test (ConCloze) is far and few between. This is despite the fact that it has a potential for item writing without the aid of native speakers and for making use of corpus-based technology in task design. This article explores the test-taking processes and strategies of a ConCloze item variant. The aim is to investigate the substantive aspect of the construct validity for the item type and increase the generalizability of the findings in the universe of admissible observations. The sample consists of 14 non-native English users who are in higher education and engage with 3 test tasks each, totaling 42 verbal reports. The sampling method is purposive sampling, in which their first language profiles are maximized in terms of heterogeneity for increased power of generalization. It is found that Reading concordance lines and recognizing clue words inside is a major process, and Assessing item components and testing a meaningful compatibility of a word in context a secondary one. A model of communicative language ability is used to provide a context for task use in this study, where strategic competence is represented in ConCloze substantive validity.

Spousal Communication During Military Deployments

2016 ◽  
Vol 37 (16) ◽  
pp. 2309-2332 ◽  
Author(s):  
Sarah P. Carter ◽  
Keith D. Renshaw
Keyword(s):  
Empirical Studies ◽  
Communication Technologies ◽  
Emerging Technology ◽  
Service Members ◽  
Military Deployments ◽  
Long Distance ◽  
Spousal Communication ◽  
Long Distance Relationships ◽  
Video Calling ◽  
Research Recommendations

Military deployments are stressful for service members and partners. Communication is an important factor in trying to maintain a relationship during these separations. This article presents a brief overview of communication in long-distance relationships for context, then reviews articles on communication during military deployments. This review reveals that emerging technology has resulted in an increase in the ability to communicate during deployment, although some studies suggest that access to such technology may vary. The few empirical studies that examine new communication technologies have found that different media (e.g., video calling vs. letters) may serve different functions in communication during deployment (e.g., facilitating problem discussion vs. providing tangible reminders of the partner). Military specific concerns, such as restrictions on communication and the potential for communication to distract service members from their mission, also appear to be important factors. The article concludes with clinical and research recommendations.

scholarly journals Assessments on Vocational Knowledge and Skills: A Content Validity Analysis

2021 ◽  
Vol 10 (3) ◽  
pp. 1529-1540
Author(s):  
Mazlili Suhaini ◽  
Adnan Ahmad ◽  
Normila Mohd
Keyword(s):  
Content Validity ◽  
Teaching And Learning ◽  
Empirical Studies ◽  
Item Analysis ◽  
Education Institution ◽  
Content Validation ◽  
Knowledge And Skills ◽  
Validity Analysis ◽  
Item Development ◽  
Alternative Research

<p style="text-align: justify;">While previous studies on assessments focused on measurement of principles and constructs, existing attention is given to content validation involving vocational education and skills. The emphasis on content validation has prompted a holistic perspective of teaching and learning to demand alternative research approaches. Using evidence of content validity based on expert judgment and assessment items, this paper argues that vocational knowledge and skills could be determined via constructs and construct functions. Content validity analysis was studied in two major phases, namely, through an assessment of an electrical technology course at a Malaysian higher education institution and expert panels’ examination of items. It was found that to increase confidence in providing reliable instruments for future empirical studies, a careful process in item development and content validity analysis was considered important. Therefore, applying these findings on item analysis and expert panels to reflect content validity can enhance the validity of assessment items.</p>

Achievement Tests in Primary Mathematics: Perpetuating Lower-Order Thinking

1991 ◽  
Vol 38 (9) ◽  
pp. 4-9 ◽  
Author(s):  
Constance Kamii ◽  
Barbara Ann Lewis
Keyword(s):  
Mathematics Education ◽  
Research Council ◽  
National Research Council ◽  
Current Method ◽  
Achievement Tests ◽  
Achievement Testing ◽  
Evaluation Standards ◽  
Standardized Achievement Tests ◽  
Method Of Evaluation ◽  
Harmful Effects

The Curriculum and Evaluation Standards (NCTM 1989) states that if we want to improve the nation's mathematics education, it is necessary to change the current method of evaluation that depends on standardized achievement tests. The National Research Council (1989) is even more explicit about the harmful effects of achievement testing.

ITEM WRITING IN EMPIRICAL STUDIES

1969 ◽  
Vol 11 (3) ◽  
pp. 223-225
Author(s):  
Eric Hall
Keyword(s):  
Empirical Studies ◽  
Item Writing

scholarly journals Development of Items Designed to Evaluate Activity Performance and Participation in Children and Adolescents with Spinal Cord Injury

2009 ◽  
Vol 2009 ◽  
pp. 1-7 ◽  
Author(s):  
Christina L. Calhoun ◽  
Stephen M. Haley ◽  
Anne Riley ◽  
Lawrence C. Vogel ◽  
Craig M. McDonald ◽  
...  
Keyword(s):  
Spinal Cord ◽  
Spinal Cord Injury ◽  
Children And Adolescents ◽  
Assessment Tools ◽  
Construct Development ◽  
Item Development ◽  
Cord Injury ◽  
Activity Performance ◽  
Item Writing ◽  
Wheeled Mobility

Background/Objective. Outcomes-based data, whether used clinically or for research, are difficult to collect in the pediatric spinal cord injury (SCI) population due to a lack of appropriate assessment measures. The purpose of this paper is twofold: to describe the process by which two item pools were developed to evaluate activity performance and participation among children with SCI and to introduce the resultant items specific to pediatric SCI.Methods. The process of item development, including construct development, review of related assessment tools, chart review, item writing and refinement using focus groups, cognitive interviews, and further refinement, was used to create the items pools for activity and participation for children and adolescents with SCI.Results. A total of 347 items were written for the activity performance construct and 61 items were written for the participation construct. Several domains were established within each construct and items were written for both child and parent respondents.Conclusion. The process of detailed item development is the first step in the process of developing an outcomes instrument for children and adolescents with SCI to assess activity performance and participation. The items are representative of pediatric SCI because they address areas specific to children and adolescents with SCI such as wheeled mobility, upper extremity function with adaptive equipment, role performance, and socialization. After testing these items in calibration studies, we will determine if these items can be developed into effective computer-adaptive testing applications.

Dynamic Assessment Language Tasks and the Prediction of Performance on Year-End Language Skills in Preschool Dual Language Learners

2020 ◽  
Vol 29 (3) ◽  
pp. 1226-1240
Author(s):  
Janet L. Patterson ◽  
Barbara L. Rodríguez ◽  
Philip S. Dale
Keyword(s):  
Head Start ◽  
Language Learners ◽  
Language Impairment ◽  
Dual Language ◽  
Dynamic Assessment ◽  
Dual Language Learners ◽  
Achievement Testing ◽  
Potential Applications ◽  
Language Screening ◽  
Relationship Of

Purpose Early identification is a key element for accessing appropriate services for preschool children with language impairment. However, there is a high risk of misidentifying typically developing dual language learners as having language impairment if inappropriate tools designed for monolingual children are used. In this study of children with bilingual exposure, we explored performance on brief dynamic assessment (DA) language tasks using graduated prompting because this approach has potential applications for screening. We asked if children's performance on DA language tasks earlier in the year was related to their performance on a year-end language achievement measure. Method Twenty 4-year-old children from Spanish-speaking homes attending Head Start preschools in the southwestern United States completed three DA graduated prompting language tasks 3–6 months prior to the Head Start preschools' year-end achievement testing. The DA tasks, Novel Adjective Learning, Similarities in Function, and Prediction, were administered in Spanish, but correct responses in English or Spanish were accepted. The year-end achievement measure, the Learning Accomplishment Profile–Third Edition (LAP3), was administered by the children's Head Start teachers, who also credited correct responses in either language. Results Children's performance on two of the three DA language tasks was significantly and positively related to year-end LAP3 language scores, and there was a moderate and significant relationship for one of the DA tasks, even when controlling for age and initial LAP3 scores. Conclusions Although the relationship of performance on DA with year-end performance varies across tasks, the findings indicate potential for using a graduated prompting approach to language screening with young dual language learners. Further research is needed to select the best tasks for administration in a graduated prompting framework and determine accuracy of identification of language impairment.

An empirical review of gambling expansion and gambling-related harm

2018 ◽  
Vol 64 (5-6) ◽  
pp. 295-306 ◽  
Author(s):  
Debi A. LaPlante ◽  
Heather M. Gray ◽  
Pat M. Williams ◽  
Sarah E. Nelson
Keyword(s):  
Literature Review ◽  
Peer Review ◽  
Empirical Data ◽  
Empirical Studies ◽  
Expansion Method ◽  
Responsible Gambling ◽  
Review Literature ◽  
Evidence Based ◽  
Empirical Comparison ◽  
Related Harm

Abstract. Aims: To discuss and review the latest research related to gambling expansion. Method: We completed a literature review and empirical comparison of peer reviewed findings related to gambling expansion and subsequent gambling-related changes among the population. Results: Although gambling expansion is associated with changes in gambling and gambling-related problems, empirical studies suggest that these effects are mixed and the available literature is limited. For example, the peer review literature suggests that most post-expansion gambling outcomes (i. e., 22 of 34 possible expansion outcomes; 64.7 %) indicate no observable change or a decrease in gambling outcomes, and a minority (i. e., 12 of 34 possible expansion outcomes; 35.3 %) indicate an increase in gambling outcomes. Conclusions: Empirical data related to gambling expansion suggests that its effects are more complex than frequently considered; however, evidence-based intervention might help prepare jurisdictions to deal with potential consequences. Jurisdictions can develop and evaluate responsible gambling programs to try to mitigate the impacts of expanded gambling.

Investigating the Multimodal Nature of Human Communication

2009 ◽  
Vol 23 (2) ◽  
pp. 63-76 ◽  
Author(s):  
Silke Paulmann ◽  
Sarah Jessen ◽  
Sonja A. Kotz
Keyword(s):  
Visual Information ◽  
Empirical Studies ◽  
Physical Property ◽  
Event Related Potentials ◽  
Accurate Information ◽  
Human Communication ◽  
Channel Condition ◽  
Prosodic Cues ◽  
Early Processing ◽  
Related Potentials

The multimodal nature of human communication has been well established. Yet few empirical studies have systematically examined the widely held belief that this form of perception is facilitated in comparison to unimodal or bimodal perception. In the current experiment we first explored the processing of unimodally presented facial expressions. Furthermore, auditory (prosodic and/or lexical-semantic) information was presented together with the visual information to investigate the processing of bimodal (facial and prosodic cues) and multimodal (facial, lexic, and prosodic cues) human communication. Participants engaged in an identity identification task, while event-related potentials (ERPs) were being recorded to examine early processing mechanisms as reflected in the P200 and N300 component. While the former component has repeatedly been linked to physical property stimulus processing, the latter has been linked to more evaluative “meaning-related” processing. A direct relationship between P200 and N300 amplitude and the number of information channels present was found. The multimodal-channel condition elicited the smallest amplitude in the P200 and N300 components, followed by an increased amplitude in each component for the bimodal-channel condition. The largest amplitude was observed for the unimodal condition. These data suggest that multimodal information induces clear facilitation in comparison to unimodal or bimodal information. The advantage of multimodal perception as reflected in the P200 and N300 components may thus reflect one of the mechanisms allowing for fast and accurate information processing in human communication.

Handwriting Analysis and Personality Assessment

2000 ◽  
Vol 5 (1) ◽  
pp. 44-51 ◽  
Author(s):  
Peter Greasley
Keyword(s):  
Personality Traits ◽  
Personality Assessment ◽  
Personnel Selection ◽  
Empirical Studies ◽  
Handwriting Analysis ◽  
Recruitment Process ◽  
Academic Literature ◽  
Personnel Recruitment ◽  
The Subject ◽  
Better Than

It has been estimated that graphology is used by over 80% of European companies as part of their personnel recruitment process. And yet, after over three decades of research into the validity of graphology as a means of assessing personality, we are left with a legacy of equivocal results. For every experiment that has provided evidence to show that graphologists are able to identify personality traits from features of handwriting, there are just as many to show that, under rigorously controlled conditions, graphologists perform no better than chance expectations. In light of this confusion, this paper takes a different approach to the subject by focusing on the rationale and modus operandi of graphology. When we take a closer look at the academic literature, we note that there is no discussion of the actual rules by which graphologists make their assessments of personality from handwriting samples. Examination of these rules reveals a practice founded upon analogy, symbolism, and metaphor in the absence of empirical studies that have established the associations between particular features of handwriting and personality traits proposed by graphologists. These rules guide both popular graphology and that practiced by professional graphologists in personnel selection.

Sign in / Sign up

Export Citation Format

Share Document