scholarly journals SMS feedback system as a quality assurance mechanism: experience from a household survey in rural India

2021 ◽  
Vol 6 (Suppl 5) ◽  
pp. e005287
Author(s):  
Neha Shah ◽  
Osama Ummer ◽  
Kerry Scott ◽  
Jean Juste Harrisson Bashingwa ◽  
Nehru Penugonda ◽  
...  
Keyword(s):  
Quality Assurance ◽  
Data Quality ◽  
Health Workers ◽  
Digital Health ◽  
Qualitative Interviews ◽  
Household Survey ◽  
Feedback Loops ◽  
Text Messages ◽  
Data Capture ◽  
Quality Assurance Mechanism

The increasing use of digital health solutions to support data capture both as part of routine delivery of health services and through special surveys presents unique opportunities to enhance quality assurance measures. This study aims to demonstrate the feasibility and acceptability of using back-end data analytics and machine learning to identify impediments in data quality and feedback issues requiring follow-up to field teams using automated short messaging service (SMS) text messages. Data were collected as part of a postpartum women’s survey (n=5095) in four districts of Madhya Pradesh, India, from October 2019 to February 2020. SMSs on common errors found in the data were sent to supervisors and coordinators. Before/after differences in time to correction of errors were examined, and qualitative interviews conducted with supervisors, coordinators, and enumerators. Study activities resulted in declines in the average number of errors per week after the implementation of automated feedback loops. Supervisors and coordinators found the direct format, complete information, and automated nature of feedback convenient to work with and valued the more rapid notification of errors. However, coordinators and supervisors reported preferring group WhatsApp messages as compared with individual SMSs to each supervisor/coordinator. In contrast, enumerators preferred the SMS system over in-person group meetings where data quality impediments were discussed. This study demonstrates that automated SMS feedback loops can be used to enhance survey data quality at minimal cost. Testing is needed among data capture applications in use by frontline health workers in India and elsewhere globally.

scholarly journals Mobile health messaging service and helpdesk for South African mothers (MomConnect): history, successes and challenges

2018 ◽  
Vol 3 (Suppl 2) ◽  
pp. e000559 ◽  
Author(s):  
Peter Barron ◽  
Joanne Peter ◽  
Amnesty E LeFevre ◽  
Jane Sebidi ◽  
Marcha Bekker ◽  
...  
Keyword(s):  
Pregnant Women ◽  
South African ◽  
Mobile Health ◽  
Health Workers ◽  
Digital Health ◽  
Universal Coverage ◽  
Text Messages ◽  
Time Data ◽  
Department Of Health ◽  
Early Decision

MomConnect is a flagship programme of the South African National Department of Health that has reached over 1.5 million pregnant women. Using mobile technology, MomConnect provides pregnant and postpartum women with twice-weekly health information text messages as well as access to a helpdesk for patient queries and feedback. In just 3 years, MomConnect has been taken to scale to reach over 95% of public health facilities and has reached 63% of all pregnant women attending their first antenatal appointment. The helpdesk has received over 300 000 queries at an average of 250 per day from 6% of MomConnect users. The service is entirely free to its users. The rapid deployment of MomConnect has been facilitated by strong government leadership, and an ecosystem of mobile health implementers who had experience of much of the content and technology required. An early decision to design MomConnect for universal coverage has required the use of text-based technologies (short messaging service and Unstructured Supplementary Service Data) that are accessible via even the most basic mobile phones, but cumbersome to use and costly at scale. Unlike previous mobile messaging services in South Africa, MomConnect collects the user’s identification number and facility code during registration, enabling future linkages with other health and population databases and geolocated feedback. MomConnect has catalysed additional efforts to strengthen South Africa’s digital health architecture. The rapid growth in smartphone penetration presents new opportunities to reduce costs, increase real-time data collection and expand the reach and scope of MomConnect to serve health workers and other patient groups.

scholarly journals Using Machine Learning to Optimize the Quality of Survey Data: Protocol for a Use Case in India

10.2196/17619 ◽  
2020 ◽  
Vol 9 (8) ◽  
pp. e17619
Author(s):  
Neha Shah ◽  
Diwakar Mohan ◽  
Jean Juste Harisson Bashingwa ◽  
Osama Ummer ◽  
Arpita Chakraborty ◽  
...  
Keyword(s):  
Machine Learning ◽  
Quality Assurance ◽  
Data Collection ◽  
Data Quality ◽  
Survey Data ◽  
Data Analytics ◽  
Feedback Loops ◽  
Time Data ◽  
Real Time Data

Background Data quality is vital for ensuring the accuracy, reliability, and validity of survey findings. Strategies for ensuring survey data quality have traditionally used quality assurance procedures. Data analytics is an increasingly vital part of survey quality assurance, particularly in light of the increasing use of tablets and other electronic tools, which enable rapid, if not real-time, data access. Routine data analytics are most often concerned with outlier analyses that monitor a series of data quality indicators, including response rates, missing data, and reliability of coefficients for test-retest interviews. Machine learning is emerging as a possible tool for enhancing real-time data monitoring by identifying trends in the data collection, which could compromise quality. Objective This study aimed to describe methods for the quality assessment of a household survey using both traditional methods as well as machine learning analytics. Methods In the Kilkari impact evaluation’s end-line survey amongst postpartum women (n=5095) in Madhya Pradesh, India, we plan to use both traditional and machine learning–based quality assurance procedures to improve the quality of survey data captured on maternal and child health knowledge, care-seeking, and practices. The quality assurance strategy aims to identify biases and other impediments to data quality and includes seven main components: (1) tool development, (2) enumerator recruitment and training, (3) field coordination, (4) field monitoring, (5) data analytics, (6) feedback loops for decision making, and (7) outcomes assessment. Analyses will include basic descriptive and outlier analyses using machine learning algorithms, which will involve creating features from time-stamps, “don’t know” rates, and skip rates. We will also obtain labeled data from self-filled surveys, and build models using k-folds cross-validation on a training data set using both supervised and unsupervised learning algorithms. Based on these models, results will be fed back to the field through various feedback loops. Results Data collection began in late October 2019 and will span through March 2020. We expect to submit quality assurance results by August 2020. Conclusions Machine learning is underutilized as a tool to improve survey data quality in low resource settings. Study findings are anticipated to improve the overall quality of Kilkari survey data and, in turn, enhance the robustness of the impact evaluation. More broadly, the proposed quality assurance approach has implications for data capture applications used for special surveys as well as in the routine collection of health information by health workers. International Registered Report Identifier (IRRID) DERR1-10.2196/17619

scholarly journals A qualitative and quantitative performance evaluation of Swaziland’s Rural Health Motivator program

F1000Research ◽  
2017 ◽  
Vol 6 ◽  
pp. 607
Author(s):  
Pascal Geldsetzer ◽  
Maria Vaikath ◽  
Jan-Walter De Neve ◽  
Till Bärnighausen ◽  
Thomas J. Bossert
Keyword(s):  
Rural Health ◽  
Health Workers ◽  
Qualitative Interviews ◽  
Household Survey ◽  
Evaluation Framework ◽  
Simple Random Sample ◽  
Care Seeking ◽  
Community Members ◽  
Emergency Situations ◽  
Quantitative Performance

Background: Community health workers (CHWs) are increasingly used to increase access to primary healthcare, and considered to be a key health worker cadre to achieve the UNAIDS 90-90-90 target. Despite the recent policy interest in effectively designing, implementing, and evaluating new CHW programs, there is limited evidence on how long-standing CHW programs are performing. Using the CHW Performance Logic model as an evaluation framework, this study aims to assess the performance of Swaziland’s long-standing national CHW program, called the rural health motivator (RHM) program. Methods: This study was carried out in the Manzini and Lubombo regions of Swaziland. We conducted a survey of 2,000 households selected through two-stage cluster random sampling and a survey among a stratified simple random sample of 306 RHMs. Additionally, semi-structured qualitative interviews were conducted with 25 RHMs. Results: While RHMs are instructed to visit every household assigned to them at least once a month, only 15.7% (95% CI: 11.4 – 20.4%) of RHMs self-reported to be meeting this target. Less than half (46.3%; 95% CI: 43.4 – 49.6%) of household survey respondents, who reported to have ever been visited by a RHM, rated their overall satisfaction with RHM services as eight or more points on a 10-point scale (ranging from “very dissatisfied” to “very satisfied”). A theme arising from the qualitative interviews was that community members only rarely seek care from RHMs, with care-seeking tending to be constrained to emergency situations. Conclusions: The RHM program does not meet some of its key performance objectives. Two opportunities to improve RHM performance identified by the evaluation were increasing RHM's stipend and improving the supply of equipment and material resources needed by RHMs to carry out their tasks.

scholarly journals Using Machine Learning to Optimize the Quality of Survey Data: Protocol for a Use Case in India (Preprint)

2019 ◽  
Author(s):  
Neha Shah ◽  
Diwakar Mohan ◽  
Jean Juste Harisson Bashingwa ◽  
Osama Ummer ◽  
Arpita Chakraborty ◽  
...  
Keyword(s):  
Machine Learning ◽  
Quality Assurance ◽  
Data Collection ◽  
Data Quality ◽  
Survey Data ◽  
Data Analytics ◽  
Feedback Loops ◽  
Time Data ◽  
Real Time Data

BACKGROUND Data quality is vital for ensuring the accuracy, reliability, and validity of survey findings. Strategies for ensuring survey data quality have traditionally used quality assurance procedures. Data analytics is an increasingly vital part of survey quality assurance, particularly in light of the increasing use of tablets and other electronic tools, which enable rapid, if not real-time, data access. Routine data analytics are most often concerned with outlier analyses that monitor a series of data quality indicators, including response rates, missing data, and reliability of coefficients for test-retest interviews. Machine learning is emerging as a possible tool for enhancing real-time data monitoring by identifying trends in the data collection, which could compromise quality. OBJECTIVE This study aimed to describe methods for the quality assessment of a household survey using both traditional methods as well as machine learning analytics. METHODS In the Kilkari impact evaluation’s end-line survey amongst postpartum women (n=5095) in Madhya Pradesh, India, we plan to use both traditional and machine learning–based quality assurance procedures to improve the quality of survey data captured on maternal and child health knowledge, care-seeking, and practices. The quality assurance strategy aims to identify biases and other impediments to data quality and includes seven main components: (1) tool development, (2) enumerator recruitment and training, (3) field coordination, (4) field monitoring, (5) data analytics, (6) feedback loops for decision making, and (7) outcomes assessment. Analyses will include basic descriptive and outlier analyses using machine learning algorithms, which will involve creating features from time-stamps, “don’t know” rates, and skip rates. We will also obtain labeled data from self-filled surveys, and build models using k-folds cross-validation on a training data set using both supervised and unsupervised learning algorithms. Based on these models, results will be fed back to the field through various feedback loops. RESULTS Data collection began in late October 2019 and will span through March 2020. We expect to submit quality assurance results by August 2020. CONCLUSIONS Machine learning is underutilized as a tool to improve survey data quality in low resource settings. Study findings are anticipated to improve the overall quality of Kilkari survey data and, in turn, enhance the robustness of the impact evaluation. More broadly, the proposed quality assurance approach has implications for data capture applications used for special surveys as well as in the routine collection of health information by health workers. CLINICALTRIAL INTERNATIONAL REGISTERED REPORT DERR1-10.2196/17619

AVADAR Application: a learning model for designing a mobile-based disease surveillance system using participatory community reporting in Africa (Preprint)

2019 ◽  
Author(s):  
Godwin Akpan ◽  
Johnson Muluh Ticha ◽  
Lara M.F. Paige ◽  
Daniel Rasheed Oyaole ◽  
Patrick Briand ◽  
...  
Keyword(s):  
Community Participation ◽  
Surveillance System ◽  
Disease Surveillance ◽  
Health Workers ◽  
Digital Health ◽  
Black Box ◽  
Project Planning ◽  
Acceptance Testing ◽  
Surveillance Strategy ◽  
Design And Implementation

BACKGROUND Acute Flaccid Paralysis (AFP) surveillance is the bedrock of polio case detection. The Auto Visual AFP Detection and Reporting (AVADAR) is a digital health intervention designed as a supplemental community surveillance system. OBJECTIVE This paper describes the design and implementation process that made AVADAR a successful disease surveillance strategy at the community level. METHODS This paper outlines the methods for the design and implementation of the AVADAR application. It explains the co-design of the application, the implementation of a helpdesk support structure, the process involved in trouble shooting the application, the benefits of utilizing a closed user group for telecommunication requirements, and the use of a consented video. We also describe how these features combined led to user acceptance testing using black box methodology. RESULTS A total of 198 community informants across two provinces, four districts and 32 settlements were interviewed about application performance, usability, security, load, stress and functionality testing black box components. The responses showed most community participants giving positive reviews. Data from the Blackbox testing yielded optimum acceptance ratings from over 90% of the users involved in the testing. A total of 22380 AFP Alerts were sent out by community informants and 21589 (95%) were investigated by health workers or WHO AVADAR coordinators. Overall there was 93% assimilation at regional level. About 83% of investigations were done in the vicinity of the alerts in 2018 compared to 77% in 2017. CONCLUSIONS AVADAR implementation model offers a simplistic step by step model that includes community participation as an integral tool for the successful deployment of a mobile based surveillance reporting tool. AVADAR can be a veritable source of project planning data and a mobile application for other interventions that target using community participation to influence health outcomes.

scholarly journals mCARE, a digital health intervention package on pregnancy surveillance and care-seeking reminders from 2018 to 2027 in Bangladesh: a model-based cost-effectiveness analysis

BMJ Open ◽  
2021 ◽  
Vol 11 (4) ◽  
pp. e042553
Author(s):  
Youngji Jo ◽  
Amnesty Elizabeth LeFevre ◽  
Hasmot Ali ◽  
Sucheta Mehra ◽  
Kelsey Alland ◽  
...  
Keyword(s):  
Cost Effectiveness ◽  
Home Visit ◽  
Digital Health ◽  
Status Quo ◽  
Health Intervention ◽  
Text Messages ◽  
Care Seeking ◽  
Intervention Package ◽  
The Status ◽  
The Cost

ObjectiveWe estimated the cost-effectiveness of a digital health intervention package (mCARE) for community health workers, on pregnancy surveillance and care-seeking reminders compared with the existing paper-based status quo, from 2018 to 2027, in Bangladesh.InterventionsThe mCARE programme involved digitally enhanced pregnancy surveillance, individually targeted text messages and in-person home-visit to pregnant women for care-seeking reminders for antenatal care, child delivery and postnatal care.Study designWe developed a model to project population and service coverage increases with annual geographical expansion (from 1 million to 10 million population over 10 years) of the mCARE programme and the status quo.Major outcomesFor this modelling study, we used Lives Saved Tool to estimate the number of deaths and disability-adjusted life years (DALYs) that would be averted by 2027, if the coverage of health interventions was increased in mCARE programme and the status quo, respectively. Economic costs were captured from a societal perspective using an ingredients approach and expressed in 2018 US dollars. Probabilistic sensitivity analysis was undertaken to account for parameter uncertainties.ResultsWe estimated the mCARE programme to avert 3076 deaths by 2027 at an incremental cost of $43 million relative to the status quo, which is translated to $462 per DALY averted. The societal costs were estimated to be $115 million for mCARE programme (48% of which are programme costs, 35% user costs and 17% provider costs). With the continued implementation and geographical scaling-up, the mCARE programme improved its cost-effectiveness from $1152 to $462 per DALY averted from 5 to 10 years.ConclusionMobile phone-based pregnancy surveillance systems with individually scheduled text messages and home-visit reminder strategies can be highly cost-effective in Bangladesh. The cost-effectiveness may improve as it promotes facility-based child delivery and achieves greater programme cost efficiency with programme scale and sustainability.

Sign in / Sign up

Export Citation Format

Share Document