scholarly journals A comparison of holistic, analytic, and part marking models in speaking assessment

2020 ◽  
Vol 37 (3) ◽  
pp. 333-360
Author(s):  
Nahal Khabbazbashi ◽  
Evelina D. Galaczi
Keyword(s):  
Mixed Methods ◽  
Phase 1 ◽  
Mixed Methods Study ◽  
Measurement Properties ◽  
Phase 2 ◽  
Strong Correlations ◽  
Speaking Test ◽  
Speaking Assessment ◽  
Ability Levels

This mixed methods study examined holistic, analytic, and part marking models (MMs) in terms of their measurement properties and impact on candidate CEFR classifications in a semi-direct online speaking test. Speaking performances of 240 candidates were first marked holistically and by part (phase 1). On the basis of phase 1 findings—which suggested stronger measurement properties for the part MM—phase 2 focused on a comparison of part and analytic MMs. Speaking performances of 400 candidates were rated analytically and by part during that phase. Raters provided open comments on their marking experiences. Results suggested a significant impact of MM; approximately 30% and 50% of candidates in phases 1 and 2 respectively were awarded different (adjacent) CEFR levels depending on the choice of MM used to assign scores. There was a trend of higher CEFR levels with the holistic MM and lower CEFR levels with the part MM. Although strong correlations were found between all pairings of MMs, further analyses revealed important differences. The part MM was shown to display superior measurement qualities particularly in allowing raters to make finer distinctions between different speaking ability levels. These findings have implications for the scoring validity of speaking tests.

scholarly journals Codeveloping a multibehavioural mobile phone app to enhance social and emotional well-being and reduce health risks among Aboriginal and Torres Strait Islander women during preconception and pregnancy: a three-phased mixed-methods study

BMJ Open ◽  
2021 ◽  
Vol 11 (11) ◽  
pp. e052545
Author(s):  
Michelle Kennedy ◽  
Ratika Kumar ◽  
Nicole M Ryan ◽  
Jessica Bennett ◽  
Gina La Hera Fuentes ◽  
...  
Keyword(s):  
Mixed Methods ◽  
Mobile Phone ◽  
Phase 1 ◽  
Qualitative Interviews ◽  
Mixed Methods Study ◽  
Phase 2 ◽  
Community Members ◽  
Phase 3 ◽  
Aboriginal Community ◽  
Aboriginal Women

ObjectiveDescribe the development and pretest of a prototype multibehavioural change app MAMA-EMPOWER.DesignMixed-methods study reporting three phases: (1) contextual enquiry included stakeholder engagement and qualitative interviews with Aboriginal women, (2) value specification included user-workshop with an Aboriginal researcher, community members and experts, (3) codesign with Aboriginal researchers and community members, followed by a pretest of the app with Aboriginal women, and feedback from qualitative interviews and the user-Mobile Application Rating Scale (U-MARS) survey tool.SettingsAboriginal women and communities in urban and regional New South Wales, Australia.ParticipantsPhase 1: interviews, 8 Aboriginal women. Phase 2: workshop, 6 Aboriginal women. Phase 3: app trial, 16 Aboriginal women. U-MARS, 5 Aboriginal women.ResultsPhase 1 interviews revealed three themes: current app use, desired app characteristics and implementation. Phase 2 workshop provided guidance for the user experience. Phase 3 app trial assessed all content areas. The highest ratings were for information (mean score of 3.80 out of 5, SD=0.77) and aesthetics (mean score of 3.87 with SD of 0.74), while functionality, engagement and subjective quality had lower scores. Qualitative interviews revealed the acceptability of the app, however, functionality was problematic.ConclusionsDeveloping a mobile phone app, particularly in an Aboriginal community setting, requires extensive consultation, negotiation and design work. Using a strong theoretical foundation of behavioural change technique’s coupled with the consultative approach has added rigour to this process. Using phone apps to implement behavioural interventions in Aboriginal community settings remains a new area for investigation. In the next iteration of the app, we aim to find better ways to personalise the content to women’s needs, then ensure full functionality before conducting a larger trial. We predict the process of development will be of interest to other health researchers and practitioners.

scholarly journals Identification of Factors That Motivate People With Multiple Sclerosis to Participate in Digital Data Collection in Research: Sequential Mixed Methods Study (Preprint)

2019 ◽  
Author(s):  
Astrid Karnoe ◽  
Lars Kayser ◽  
Lasse Skovgaard
Keyword(s):  
Multiple Sclerosis ◽  
Content Analysis ◽  
Mixed Methods ◽  
Data Collection ◽  
Phase 1 ◽  
Mixed Methods Study ◽  
Digital Data ◽  
Phase 2 ◽  
Data Collection Tool ◽  
Sequential Mixed Methods

BACKGROUND Digital data collection has the potential to reduce participant burden in research projects that require extensive registrations from participants. To achieve this, a digital data collection tool needs to address potential barriers and motivations for participation. OBJECTIVE This study aimed to identify factors that may affect motivation for participation and adoption of a digital data collection tool in a research project on nutrition and multiple sclerosis (MS). METHODS The study was designed as a sequential mixed methods study with 3 phases. In phase 1, 15 semistructured interviews were conducted in a Danish population of individuals with MS. Interview guide frameworks were based on dimensions from the electronic health literacy framework and the Health Education Impact Questionnaire. Data from phase 1 were analyzed in a content analysis, and findings were used to inform the survey design in phase 2 that validates the results from the content analysis in a larger population. The survey consisted of 14 items, and it was sent to 1000 individuals with MS (response rate 42.5%). In phase 3, participants in 3 focus group interviews discussed how findings from phases 1 and 2 might affect motivation for participation and adoption of the digital tool. RESULTS The following 3 categories related to barriers and incentives for participation were identified in the content analysis of the 15 individual interviews: (1) life with MS, (2) use of technology, and (3) participation and incentives. Phase 1 findings were tested in phase 2’s survey in a larger population (n=1000). The majority of participants were comfortable using smartphone technologies and participated actively on social media platforms. MS symptoms did cause limitations in the use of Web pages and apps when the given pages had screen clutter, too many colors, or too small buttons. Life with MS meant that most participants had to ration their energy levels. Support from family and friends was important to participants, but support could also come in the form of physical aids (walking aids and similar) and digital aids (reminders, calendar functions, and medication management). Factors that could discourage participation were particularly related to the time it would take every day. The biggest motivations for participation were to contribute to research in MS, to learn more about one’s own MS and what affects it, and to be able to exchange experiences with other people with MS. CONCLUSIONS MS causes limitations that put demands on tools developed for digital data collection. A digital data collection tool can increase chances of high adoption rates, but it needs to be supplemented with a clear and simple project design and continuous communication with participants. Motivational factors should be considered in both study design and the development of a digital data collection tool for research.

scholarly journals Identification of Factors That Motivate People With Multiple Sclerosis to Participate in Digital Data Collection in Research: Sequential Mixed Methods Study

10.2196/13295 ◽  
2019 ◽  
Vol 6 (4) ◽  
pp. e13295 ◽  
Author(s):  
Astrid Karnoe ◽  
Lars Kayser ◽  
Lasse Skovgaard
Keyword(s):  
Multiple Sclerosis ◽  
Content Analysis ◽  
Mixed Methods ◽  
Data Collection ◽  
Phase 1 ◽  
Mixed Methods Study ◽  
Digital Data ◽  
Phase 2 ◽  
Data Collection Tool ◽  
Sequential Mixed Methods

Background Digital data collection has the potential to reduce participant burden in research projects that require extensive registrations from participants. To achieve this, a digital data collection tool needs to address potential barriers and motivations for participation. Objective This study aimed to identify factors that may affect motivation for participation and adoption of a digital data collection tool in a research project on nutrition and multiple sclerosis (MS). Methods The study was designed as a sequential mixed methods study with 3 phases. In phase 1, 15 semistructured interviews were conducted in a Danish population of individuals with MS. Interview guide frameworks were based on dimensions from the electronic health literacy framework and the Health Education Impact Questionnaire. Data from phase 1 were analyzed in a content analysis, and findings were used to inform the survey design in phase 2 that validates the results from the content analysis in a larger population. The survey consisted of 14 items, and it was sent to 1000 individuals with MS (response rate 42.5%). In phase 3, participants in 3 focus group interviews discussed how findings from phases 1 and 2 might affect motivation for participation and adoption of the digital tool. Results The following 3 categories related to barriers and incentives for participation were identified in the content analysis of the 15 individual interviews: (1) life with MS, (2) use of technology, and (3) participation and incentives. Phase 1 findings were tested in phase 2’s survey in a larger population (n=1000). The majority of participants were comfortable using smartphone technologies and participated actively on social media platforms. MS symptoms did cause limitations in the use of Web pages and apps when the given pages had screen clutter, too many colors, or too small buttons. Life with MS meant that most participants had to ration their energy levels. Support from family and friends was important to participants, but support could also come in the form of physical aids (walking aids and similar) and digital aids (reminders, calendar functions, and medication management). Factors that could discourage participation were particularly related to the time it would take every day. The biggest motivations for participation were to contribute to research in MS, to learn more about one’s own MS and what affects it, and to be able to exchange experiences with other people with MS. Conclusions MS causes limitations that put demands on tools developed for digital data collection. A digital data collection tool can increase chances of high adoption rates, but it needs to be supplemented with a clear and simple project design and continuous communication with participants. Motivational factors should be considered in both study design and the development of a digital data collection tool for research.

scholarly journals Development and Early Feasibility of Chatbots for Educating Patients With Lung Cancer and Their Caregivers in Japan: Mixed Methods Study

JMIR Cancer ◽  
10.2196/26911 ◽  
2021 ◽  
Vol 7 (1) ◽  
pp. e26911
Author(s):  
Yuki Kataoka ◽  
Tomoyasu Takemura ◽  
Munehiko Sasajima ◽  
Naoki Katoh
Keyword(s):  
Lung Cancer ◽  
Mixed Methods ◽  
Health Care Providers ◽  
Medical Information ◽  
Phase 1 ◽  
Test Phase ◽  
Mixed Methods Study ◽  
Phase 2 ◽  
Care Providers ◽  
Web Based

Background Chatbots are artificial intelligence–driven programs that interact with people. The applications of this technology include the collection and delivery of information, generation of and responding to inquiries, collection of end user feedback, and the delivery of personalized health and medical information to patients through cellphone- and web-based platforms. However, no chatbots have been developed for patients with lung cancer and their caregivers. Objective This study aimed to develop and evaluate the early feasibility of a chatbot designed to improve the knowledge of symptom management among patients with lung cancer in Japan and their caregivers. Methods We conducted a sequential mixed methods study that included a web-based anonymized questionnaire survey administered to physicians and paramedics from June to July 2019 (phase 1). Two physicians conducted a content analysis of the questionnaire to curate frequently asked questions (FAQs; phase 2). Based on these FAQs, we developed and integrated a chatbot into a social network service (phase 3). The physicians and paramedics involved in phase I then tested this chatbot (α test; phase 4). Thereafter, patients with lung cancer and their caregivers tested this chatbot (β test; phase 5). Results We obtained 246 questions from 15 health care providers in phase 1. We curated 91 FAQs and their corresponding responses in phase 2. In total, 11 patients and 1 caregiver participated in the β test in phase 5. The participants were asked 60 questions, 8 (13%) of which did not match the appropriate categories. After the β test, 7 (64%) participants responded to the postexperimental questionnaire. The mean satisfaction score was 2.7 (SD 0.5) points out of 5. Conclusions Medical staff providing care to patients with lung cancer can use the categories specified in this chatbot to educate patients on how they can manage their symptoms. Further studies are required to improve chatbots in terms of interaction with patients.

scholarly journals P.024 Educational needs in migraine care: results from a mixed-methods study among Canadian primary care providers and specialists

2018 ◽  
Vol 45 (s2) ◽  
pp. S22-S22
Author(s):  
S Peloquin ◽  
E Leroux ◽  
G Shapero ◽  
S Labbe ◽  
S Murray ◽  
...  
Keyword(s):  
Mixed Methods ◽  
General Practitioners ◽  
Online Survey ◽  
Medical Training ◽  
Phase 1 ◽  
Primary Care Providers ◽  
Educational Needs ◽  
Phase 2 ◽  
Care Providers ◽  
Structured Interviews

Background: Migraines are sub-optimally treated, affect millions of Canadians, and are underrepresented in medical training. A study was conducted to identify the needs of Canadian Healthcare Providers (HCPs) for migraine education, with the aim to inform the development of learning activities. Methods: This ethics-approved study was deployed in two consecutive phases using a mixed-methods approach. Phase 1 (qualitative) explored the causes of challenges to migraine care via a literature review, input from an expert working group, and semi-structured interviews with multiple stakeholders. Phase 2 (quantitative) validated these causes using an online survey. Results: The study included 103 participants (28 in phase 1; 75 in phase 2): general practitioners=37; neurologists=24; nurses=14; pharmacists=20; administrators, policy influencers and payers=8. Four areas of sub-optimal knowledge were identified: (1) Canadian guidelines, (2) diagnostic criteria, (3) preventive treatment, and (4) non-pharmacological therapies. Attitudinal issues related to the management of migraine patients were also identified. Detailed data including the frequencies of knowledge gaps among general practitioners and general neurologists will be presented along with qualitative findings. Conclusions: Educational activities for general practitioners and general neurologists who treat patients with migraines should be designed to address the four educational needs described in this study.

Regulation of Venture Capital in the EU and East and West Africa: Impacts and Possibilities

2019 ◽  
Vol 27 (2) ◽  
pp. 292-307
Author(s):  
Jean Michel Marone
Keyword(s):  
Mixed Methods ◽  
West Africa ◽  
Venture Capital ◽  
Investor Protection ◽  
Mixed Methods Study ◽  
Capital Regulation ◽  
African Continent ◽  
Strong Correlations ◽  
The Subject ◽  
The Eu

This article provides a discussion of venture capital regulation in Africa and Europe. It is based on a mixed-methods study of three representative cases across two geopolitical entities. The cases are Germany, Kenya and Nigeria. The article notes the lacuna in scholarly circles on the African continent with respect to venture capital and therefore lays the platform for robust engagement with the subject. Using analysis from Germany, it observes that lessons can be drawn on how effective institutions may not be a hindrance to capital but serve to continue to attract. With strong correlations between capital and investor protection as well as capital and taxation levels, a case is made for better institutions even in Eastern and West Africa.

A Mixed Methods–Grounded Theory Design for Producing More Refined Theoretical Models

2020 ◽  
Vol 15 (1) ◽  
pp. 61-86 ◽  
Author(s):  
Minjung Shim ◽  
Burke Johnson ◽  
Joke Bradt ◽  
Susan Gasson
Keyword(s):  
Grounded Theory ◽  
Mixed Methods ◽  
Mixed Methods Research ◽  
Phase 1 ◽  
Theory Development ◽  
Theoretical Models ◽  
Phase 2 ◽  
Meta Model ◽  
Meta Modeling ◽  
Confirmatory Phase

Current literature lacks explication of how traditional grounded theory and mixed methods–grounded theory (MM-GT) are similar/different and specific explication of how to construct MM-GT designs—our purpose is to do this. We illustrate the design process using a published study. Exploratory Phase 1 involves creation of a formative–theoretical model based on multiple implicit or explicit models identified in the literature, which are then combined into a single model using meta-modeling integration. Also, in Phase 1, a traditional grounded theory is developed “independently” using interview data. These two models are integrated into a combined/meta-model at the end of Phase 1. Confirmatory Phase 2 involves testing of the final Phase 1 meta-model using a mixed methods experiment. In Phase 3, the Phase 1 and Phase 2 results are integrated, producing the “final” meta-model. This article contributes to the field of mixed methods research by showing how to design an MM-GT study that is focused on theory development and testing.

scholarly journals An integrative counselling program to promote active ageing for older people in Thai nursing homes: an intervention mixed methods design

2021 ◽  
Vol 9 (1) ◽  
Author(s):  
Jantana Juthavantana ◽  
Nanchatsan Sakunpong ◽  
Ujsara Prasertsin ◽  
Monthira Charupheng ◽  
Sheibon Hassakama Lau
Keyword(s):  
Nursing Home ◽  
Mixed Methods ◽  
Nursing Homes ◽  
Older People ◽  
Phase 1 ◽  
The Elderly ◽  
Active Ageing ◽  
Phase 2 ◽  
Mixed Methods Design ◽  
Counselling Program

Abstract Background Active ageing has been a rapidly developing field of study in light of the growing population of older people. Acknowledgement of the lack of a counselling program to promote active ageing for the older people in nursing homes led to the development of this study which aims to investigate active ageing of the Thai elderly in a nursing home in addition to promoting active ageing for them through integrative counselling. Methods The study was conducted in a nursing home in Samut Prakan province, Thailand. The integrative counselling program referred to appropriate literature along with implementation of the Satir Model and Motivational Interviewing techniques. An intervention mixed methods design was applied in the study, which consisted of two phases. Phase 1 involved an investigation of the concept of active ageing, based on the context of older people in nursing homes by way of in-depth interviews, involving 5 participants. Phase 2 comprised of an investigation of the effects of an integrative counselling program to promote active ageing for older people in the same nursing home. There was a total of 16 participants in phase 2 which were divided equally into experimental and control groups respectively. Results Phase 1 of the study showcased qualitative results of the progress of active ageing development in older people that resulted in 4 sub-themes (Health development, spiritual development, active engagement and psychosocial support). Two parameters were used to analyze the results in phase 2. The quantitative results showed that the active ageing score of participants in the experimental arm increased significantly after enrollment (p < 0.05). Furthermore, the experimental group had a higher overall active ageing score in comparison to the control group. Qualitative results of phase 2 elicited factors promoting active ageing in the elderly which included activities, group facilitator and group atmosphere. Both quantitative and qualitative results of phase 2 proved to be significant, showing that the program managed to develop active ageing in participants. Conclusion Psychologists and multidisciplinary teams looking after older people in nursing homes are able to use this integrative counselling program for development of active ageing in the elderly population.

scholarly journals Exploring the causes and impacts of falls among ambulators with spinal cord injury using photovoice: a mixed-methods study

BMJ Open ◽  
2020 ◽  
Vol 10 (8) ◽  
pp. e039763 ◽  
Author(s):  
Hardeep Singh ◽  
Arielle Shibi Rosen ◽  
Geoff Bostick ◽  
Anita Kaiser ◽  
Kristin E Musselman
Keyword(s):  
Spinal Cord ◽  
Spinal Cord Injury ◽  
Mixed Methods ◽  
Focus Group ◽  
Fall Prevention ◽  
Fall Risk ◽  
Phase 1 ◽  
Phase 2 ◽  
Prevention Strategies ◽  
Cord Injury

ObjectivesThis study explored: (1) fall circumstances experienced by ambulators with spinal cord injury (SCI) over a 6-month period, (2) the impacts of falls-related injuries and fall risk and (3) their preferences/recommendations for fall prevention.DesignA sequential explanatory mixed-methods design with two phases.SettingA Canadian SCI rehabilitation hospital and community setting.ParticipantsThirty-three ambulators with SCI participated in phase 1 and eight participants that fell in phase 1 participated in phase 2.MethodsIn phase 1, fall circumstances were tracked using a survey that was completed each time a participant fell during the 6-month tracking period. Phase 2 involved photovoice; participants took photographs of factors that influenced their fall risk and how their fall risk impacted their work/recreational activities. Participants discussed the photographs and topics related to fall prevention in an individual interview and a focus group.ResultsOf the 33 participants, 21 fell in 6 months. Falls commonly occurred in the home while participants were changing positions or walking. Most falls occurred in the morning or afternoon. In phase 2, interviews and focus group discussion revealed three themes: (1) falls are caused by bodily impairments (eg, impaired reactive response during slips and trips and weakness and altered sensation in legs/feet), (2) impacts of fall-related injuries and fall risk (eg, psychosocial effects of fall-related injuries, limiting community participation due to the risk of falling and activity-dependent concern of falling) and (3) approaches to fall prevention (eg, fall prevention strategies used, components of fall prevention and utility of professional fall prevention strategies/interventions).ConclusionsFall prevention interventions/strategies should focus on minimising a person’s fall risk within their home as most falls occurred in the home environment. Ambulators with SCI would benefit from education and awareness about common fall circumstances that they may encounter in their daily lives.

Sign in / Sign up

Export Citation Format

Share Document