scholarly journals Exploring Challenges of Assessment in Engineering Problem Solving, A Thematic Analysis

2018 ◽  
Author(s):  
Bahar Memarian ◽  
Susan McCahan
Keyword(s):  
Thematic Analysis ◽  
Engineering Problem ◽  
Formative Feedback ◽  
Summative Evaluation ◽  
Assessment Practices ◽  
Frequency Count ◽  
Engineering Problems ◽  
Current Assessment ◽  
Insufficient Time ◽  
Survey Responses

Through a thematic analysis of open-ended survey responses, we explore the challenges assessors face during assessment (summative evaluation and formative feedback) of students’ work. The assessors were primed by asking them to mark solutions to closed-ended engineering problems typical in introductory Electrical and Civil engineering courses. The subsequent survey asked the assessors to reflect on their experience. The analysis of the thematic frequency count is presented and discussed in this paper. The data suggests that from the perspective of assessors, current assessment practices need improvement. In particular, it appears that summative evaluation is seen as the primary goal and feedback is given as time allows. However, the participants also identified insufficient time as a challenge, which may further diminish feedback

Enhancing formative feedback in orthopaedic training: Development and implementation of a competency-based assessment framework

2019 ◽  
Author(s):  
Natalie Wagner ◽  
Anita Acai ◽  
Sydney A. McQueen ◽  
Com McCarthy ◽  
Andrew McGuire ◽  
...  
Keyword(s):  
Intraclass Correlation ◽  
Formative Feedback ◽  
Assessment Practices ◽  
Tool Development ◽  
Assessment Framework ◽  
Educational Impact ◽  
Postgraduate Training Program ◽  
Current Assessment ◽  
Competency Based ◽  
Orthopaedic Training

Objective: The purpose of this study was to develop, implement, and evaluate the effectiveness of an assessment framework aimed at improving formative feedback practices in a Canadian orthopaedic postgraduate training program. Methods: Tool development began in 2014 and took place in 4 phases, each building upon the previous and informing the next. The reliability, validity, and educational impact of the tools were assessed on an ongoing basis, and changes were made accordingly. Results: One hundred eighty-two tools were completed and analyzed during the study period. Quantitative results suggested moderate to excellent agreement between raters (intraclass correlation coefficient = 0.54-0.93), and an ability of the tools to discriminate between learners at different stages of training (p’s < 0.05). Qualitative data suggested that the tools improved both the quality and quantity of formative feedback given by assessors and had begun to foster a culture change around assessment in the program. Conclusions: The tool development, implementation, and evaluation processes detailed in this article can serve as a model for other training programs to consider as they move towards adopting competency-based approaches and refining current assessment practices.

Usability Analysis of Freeform Marking on Engineering Problem Solving

2018 ◽  
Vol 62 (1) ◽  
pp. 354-358
Author(s):  
Bahar Memarian ◽  
Susan McCahan
Keyword(s):  
Problem Solving ◽  
Survey Data ◽  
Engineering Problem ◽  
Formative Feedback ◽  
First Year ◽  
Engineering Problems ◽  
Usability Analysis ◽  
Assessment Time ◽  
Group 2 ◽  
Group 1

Freeform comments as a means of providing formative feedback on engineering problems, from the perspective of feedback providers (i.e. assessors), is examined. The aim of this research is to collect and analyze assessors’ ratings on the usability of this type of task. Two course topics with different error loads in the sample solutions were used as the basis for the work. Assessors were divided into two groups: Group 1 received an evaluation package containing first year mechanics students’ test solutions with a high error load (Error1=32), while Group 2 received first year circuits students’ test solutions with low error load (Error2=11). Assessment time was held constant (ttot=20min). A standard instrument for usability was utilized. Analysis of the survey data from the assessors (n1=11, n2=19) revealed some significant differences between the two groups. In particular, Group 1 reported a lower degree of perceived consistency in marking relative to Group 2.

scholarly journals Formative Assessment Conceptualizations in Higher Education: Thematic Analysis Results

2021 ◽  
Vol 30 (11) ◽  
pp. 96-109
Author(s):  
I. V. Nikitin ◽  
A. K. Belolutskaya
Keyword(s):  
Formative Assessment ◽  
Thematic Analysis ◽  
Focal Point ◽  
Formative Feedback ◽  
Assessment Practices ◽  
Research Papers ◽  
Educational Innovations ◽  
English Writing ◽  
Assessment Procedures ◽  
Russian Research

Although assessment is integral to education, it is quite uncommon to be a focal point of complex educational innovations. Unlike schools, which employ a range of continuous formative assessment procedures, universities commonly rely on summative methods that are likely to be discrete and reluctant to students’ needs and experiences. Based on the study of 20 foreign and Russian research papers, this article analyzes the formative assessment practices that are found to be employed in universities. The analysis provides the results that shows the gaps between formative assessment concepts employed by Russian and English-writing authors. The study concludes that the two conceptualizations differ most prominently in how formativesummative procedures are divided, the formative feedback is accomplished, and participativity is provided.

Model-Eliciting Activities: A Research-Based Method for Inquiry Learning and Professional Development in the Engineering Classroom

2008 ◽  
Author(s):  
Tamara J. Moore
Keyword(s):  
Complex Systems ◽  
Student Development ◽  
Engineering Problem ◽  
First Year ◽  
Foundation Engineering ◽  
Underrepresented Populations ◽  
Model Eliciting Activities ◽  
Engineering Problems ◽  
The Given ◽  
Mathematics And Science

Attracting students to engineering is a challenge. In addition, ABET requires that engineering graduates be able to work on multi-disciplinary teams and apply mathematics and science when solving engineering problems. One manner of integrating teamwork and engineering contexts in a first-year foundation engineering course is through the use of Model-Eliciting Activities (MEAs) — realistic, client-driven problems based on the models and modeling theoretical framework. A Model-Eliciting Activity (MEA) is a real-world client-driven problem. The solution of an MEA requires the use of one or more mathematical or engineering concepts that are unspecified by the problem — students must make new sense of their existing knowledge and understandings to formulate a generalizable mathematical model that can be used by the client to solve the given and similar problems. An MEA creates an environment in which skills beyond mathematical abilities are valued because the focus is not on the use of prescribed equations and algorithms but on the use of a broader spectrum of skills required for effective engineering problem-solving. Carefully constructed MEAs can begin to prepare students to communicate and work effectively in teams; to adopt and adapt conceptual tools; to construct, describe, and explain complex systems; and to cope with complex systems. MEAs provide a learning environment that is tailored to a more diverse population than typical engineering course experiences as they allow students with different backgrounds and values to emerge as talented, and that adapting these types of activities to engineering courses has the potential to go beyond “filling the gaps” to “opening doors” to women and underrepresented populations in engineering. Further, MEAs provide evidence of student development in regards to ABET standards. Through NSF-funded grants, multiple MEAs have been developed and implemented with a MSE-flavored nanotechnology theme. This paper will focus on the content, implementation, and student results of one of these MEAs.

scholarly journals Attribution, referencing and commencing HE students as novice academic writers: Giving them more time to ‘get it’

2016 ◽  
Vol 7 (2) ◽  
pp. 43-49
Author(s):  
John Hamilton
Keyword(s):  
Academic Writing ◽  
Teaching Staff ◽  
Assessment Practices ◽  
Education Students ◽  
Academic Teaching ◽  
Current Assessment ◽  
Teaching Learning ◽  
Formative Learning ◽  
Concept Framework ◽  
Considerable Detail

The requirement for commencing higher education students to apply principles of attribution in their early academic writing frequently creates frustration both for students and academic teaching staff. Teachers often provide information on the necessity of attribution, and considerable detail on the mechanics of how to reference, and express frustration at the failure of some students to demonstrate this in their writing. In turn, many students appear overwhelmed and confused by the expectations placed on them as early academic writers. This paper explores these expectations and questions current assessment practices, advocating a longer period of formative learning before students are required to competently and accurately apply attribution principles and referencing conventions in their writing. Using the threshold concept framework (Meyer & Land, 2005), it suggests viewing attribution as a ‘conceptual gateway’ through which students must pass in becoming academic writers, and explores some implications of this for teaching, learning and assessment.

Technology-Assisted Formative Assessment

2015 ◽  
pp. 380-398 ◽  
Author(s):  
Karen E. Irving
Keyword(s):  
Formative Assessment ◽  
Teaching And Learning ◽  
Formative Feedback ◽  
First Century ◽  
Assessment Process ◽  
Just In Time ◽  
Assessment Practices ◽  
Present Information ◽  
Electronic Technologies ◽  
Share Information

Reliable just-in-time assessments are the foundation of informed teaching and learning. Modern electronic technologies assist in the formative assessment process by supporting classroom environments that allow students and teachers to assess learning and providing mechanisms to present information about student learning during instructional sequences. To implement formative assessment practices, students and teachers benefit from rich educational tasks that invite students to share information about their understanding of the lesson while the lesson is occurring in order to nurture productive learning by both teacher and student. Formative feedback is facilitated by technologies such as connected classrooms, videography, online formative quizzes, and manuscript multi-draft editing. Technology-assisted formative assessment represents a powerful option to promote improved classroom communications that support formative assessment practices for teachers in twenty-first century classrooms.

Authentic Assessment as a Tool to Enhance Student Learning in a Higher Education Institution

2020 ◽  
pp. 256-271
Author(s):  
Abatihun Alehegn Sewagegn ◽  
Boitumelo Molebogeng Diale
Keyword(s):  
Higher Education ◽  
Real Life ◽  
Authentic Assessment ◽  
Education Institution ◽  
Assessment Methods ◽  
Assessment Practices ◽  
Higher Education Institution ◽  
Traditional Assessment ◽  
Qualitative Research Design ◽  
Current Assessment

Authentic assessment plays a great role in enhancing students' learning and makes them competent in their study area. Studies indicate that assessment is authentic when the tasks have real-life value and students perform real-world tasks. Therefore, this chapter shows how lecturers practice authentic assessment to enhance students' learning in a higher education institution. To achieve this, the authors used a phenomenological qualitative research design. An interview was used to collect data. The result indicated that lecturers are highly dependent upon traditional assessment methods, which have no significant contribution to the competency of students. The practice of authentic assessment methods as a tool to enhance students' learning is limited. Therefore, the authors can conclude that enhancing students' learning using authentic assessment in their study areas is untenable if the lecturers continue to utilize their current assessment practices.

scholarly journals ‘Clients. Outdoors. Animals.’: retaining vets in UK farm animal practice—thematic analysis of free-text survey responses

2018 ◽  
Vol 184 (4) ◽  
pp. 121-121 ◽  
Author(s):  
Katherine E Adam ◽  
Sarah Baillie ◽  
Jonathan Rushton
Keyword(s):  
Rural Communities ◽  
Thematic Analysis ◽  
Career Choices ◽  
Animal Health ◽  
Farm Animal ◽  
Free Text ◽  
Farm Work ◽  
Outdoor Work ◽  
Survey Responses ◽  
Management Issues

Retaining vets in farm practice has been identified as a key strategy to maintain an adequately trained and experienced workforce to provide animal health services for livestock enterprises and government. This qualitative study aimed to explore vets’ experiences of UK farm animal practice and their perceptions of the factors that influenced their career choices. Thematic analysis of free-text survey responses from 187 vets working in farm practice and 141 who had given up farm work identified four main themes: affect (experiences of feeling or emotions), personal life, the job and the bigger picture. Those who stayed in farm practice described satisfaction with their career and enjoyment of physical, outdoor work in rural communities. Choosing to give up farm work was influenced by both personal and professional circumstances and related frequently to management issues in practice. Veterinary businesses also face challenges from the broader agricultural and veterinary sectors that affect their ability to support and retain vets. The findings presented build on previous quantitative analysis of factors associated with retention and demonstrate the complexity of individual vets’ career choices.

Sign in / Sign up

Export Citation Format

Share Document