Amazon Mechanical Turk

2012 ◽  
pp. 90-102
Author(s):  
Amber Chauncey Strain ◽  
Lucille M. Booker
Keyword(s):  
Mechanical Turk ◽  
Amazon Mechanical Turk ◽  
Quality Of Data ◽  
High Quality ◽  
Web Based ◽  
Potential Benefits ◽  
Amazon's Mechanical Turk ◽  
Short Time ◽  
Time Frames

One of the major challenges of ANLP research is the constant balancing act between the need for large samples, and the excessive time and monetary resources necessary for acquiring those samples. Amazon’s Mechanical Turk (MTurk) is a web-based data collection tool that has become a premier resource for researchers who are interested in optimizing their sample sizes and minimizing costs. Due to its supportive infrastructure, diverse participant pool, quality of data, and time and cost efficiency, MTurk seems particularly suitable for ANLP researchers who are interested in gathering large, high quality corpora in relatively short time frames. In this chapter, the authors first provide a broad description of the MTurk interface. Next, they describe the steps for acquiring IRB approval of MTurk experiments, designing experiments using the MTurk dashboard, and managing data. Finally, the chapter concludes by discussing the potential benefits and limitations of using MTurk for ANLP experimentation.

Improving Data Quality Using Amazon Mechanical Turk Through Platform Setup

2021 ◽  
pp. 193896552110254
Author(s):  
Lu Lu ◽  
Nathan Neale ◽  
Nathaniel D. Line ◽  
Mark Bonn
Keyword(s):  
Best Practices ◽  
Data Quality ◽  
Mechanical Turk ◽  
Amazon Mechanical Turk ◽  
Quality Data ◽  
High Quality ◽  
High Quality Data ◽  
Amazon's Mechanical Turk ◽  
Associated Data

As the use of Amazon’s Mechanical Turk (MTurk) has increased among social science researchers, so, too, has research into the merits and drawbacks of the platform. However, while many endeavors have sought to address issues such as generalizability, the attentiveness of workers, and the quality of the associated data, there has been relatively less effort concentrated on integrating the various strategies that can be used to generate high-quality data using MTurk samples. Accordingly, the purpose of this research is twofold. First, existing studies are integrated into a set of strategies/best practices that can be used to maximize MTurk data quality. Second, focusing on task setup, selected platform-level strategies that have received relatively less attention in previous research are empirically tested to further enhance the contribution of the proposed best practices for MTurk usage.

A Technical Guide to Using Amazon's Mechanical Turk in Behavioral Accounting Research

2017 ◽  
Vol 30 (1) ◽  
pp. 111-122 ◽  
Author(s):  
Steve Buchheit ◽  
Marcus M. Doxey ◽  
Troy Pollard ◽  
Shane R. Stinson
Keyword(s):  
Third Party ◽  
Mechanical Turk ◽  
Behavioral Accounting ◽  
Online Data ◽  
Research Participants ◽  
Traditional Research ◽  
Potential Benefits ◽  
Technical Guide ◽  
Online Data Collection ◽  
Amazon's Mechanical Turk

ABSTRACT Multiple social science researchers claim that online data collection, mainly via Amazon's Mechanical Turk (MTurk), has revolutionized the behavioral sciences (Gureckis et al. 2016; Litman, Robinson, and Abberbock 2017). While MTurk-based research has grown exponentially in recent years (Chandler and Shapiro 2016), reasonable concerns have been raised about online research participants' ability to proxy for traditional research participants (Chandler, Mueller, and Paolacci 2014). This paper reviews recent MTurk research and provides further guidance for recruiting samples of MTurk participants from populations of interest to behavioral accounting researchers. First, we provide guidance on the logistics of using MTurk and discuss the potential benefits offered by TurkPrime, a third-party service provider. Second, we discuss ways to overcome challenges related to targeted participant recruiting in an online environment. Finally, we offer suggestions for disclosures that authors may provide about their efforts to attract participants and analyze responses.

scholarly journals Improving Accuracy in Studying the Interactions of Seismic Waves with Bottom Sediments

2021 ◽  
Vol 9 (2) ◽  
pp. 229
Author(s):  
Georgy Mitrofanov ◽  
Nikita Goreyavchev ◽  
Roman Kushnarev
Keyword(s):  
Bottom Sediments ◽  
Seismic Waves ◽  
Wide Frequency Range ◽  
Quality Of Data ◽  
High Quality ◽  
Relative Characteristic ◽  
Seabed Evolution ◽  
Marine Seismic ◽  
Very High

The emerging tasks of determining the features of bottom sediments, including the evolution of the seabed, require a significant improvement in the quality of data and methods for their processing. Marine seismic data has traditionally been perceived to be of high quality compared to land data. However, high quality is always a relative characteristic and is determined by the problem being solved. In a detailed study of complex processes, the interaction of waves with bottom sediments, as well as the processes of seabed evolution over short time intervals (not millions of years), we need very high accuracy of observations. If we also need significant volumes of research covering large areas, then a significant revision of questions about the quality of observations and methods of processing is required to improve the quality of data. The article provides an example of data obtained during high-precision marine surveys and containing a wide frequency range from hundreds of hertz to kilohertz. It is shown that these data, visually having a very high quality, have variations in wavelets at all analyzed frequencies. The corresponding variations reach tens of percent. The use of the method of factor decomposition in the spectral domain made it possible to significantly improve the quality of the data, reducing the variability of wavelets by several times.

scholarly journals Rancang Bangun Sistem Informasi Penjualan Obat Berbasis Website dengan Menggunakan Metode Waterfall

2020 ◽  
Vol 6 (1) ◽  
pp. 71-82
Author(s):  
Ahmad Fauzi ◽  
Dewi Wulandari
Keyword(s):  
Information System ◽  
Data Processing ◽  
Quality Of Data ◽  
Microsoft Excel ◽  
Web Based ◽  
Drug Sales ◽  
Uml Modeling ◽  
Computer Based ◽  
A Company

Abstract: In this era of globalization, information technology is speeding up. In managing the information required good technology because the information has a greatvalue for a company. And computer technology today with its increasingly sophisticated processing speed has enabled the development of computer-based information systems. Problems that exist in Kauman Apothecary is about the data processing that is still done manually, ranging from the admission process of incoming drugs, drugs out, often the absence of matching stock between the data with the original drug, as well as in making reports that still use microsoft excel. The design of the system is described by UML modeling, drug sales information system on web-based pharmacy kauman intranet this is the best solution, can improve the quality of data processing drugs in pharmacies kauman. And with the creation of this information system, can help simplify data processing moreleverage, while keeping data safe and minimize the data kerangkapan. The design of web-based drug sales information system is made using PHP and MySQL.Keywords: Information System, Sales, Kauman PharmacyAbstrak: Dalam era globalisasi sekarang ini, teknologi informasi melaju dengan cepatnya.Dalam mengelola informasi dibutuhkan teknologi yang baik karena informasi mempunyai nilai yang besar bagi suatu perusahaan. Dan teknologi komputer sekarang ini dengan kecepatan prosesnya yang semakin canggih telah memungkinkan pengembangan sistem informasi berbasis komputer. Masalah yang ada pada Apotek Kauman yaitu mengenai pengolahan data-datanya yang masih dilakukan secara manual, mulai dari proses penerimaan obat masuk, obat keluar, sering tidak adanya kecocokan stok antara data dengan obat aslinya, serta dalam membuat laporan yang masih menggunakan microsoft excel. Perancangan sistem digambarkan dengan pemodelan UML, sistem informasi penjualan obat pada apotek kauman berbasis web intranet ini merupakan solusi yang terbaik, dapat meningkatkan kualitas pengolahan data obat di apotek kauman. Dan dengan dibuatnya sistem informasi ini, dapat membantu mempermudah pengolahan data lebih maksimal, sekaligus menjaga data tetap aman dan meminimalisir adanya kerangkapan data. Perancangan sistem informasi penjualan obat berbasis web ini dibuat menggunakan PHP dan MySQLKata Kunci: Sistem Informasi, Penjualan, Apotek Kauman.

Using MTurk to Distribute a Survey or Experiment: Methodological Considerations

2018 ◽  
Vol 33 (1) ◽  
pp. 43-65 ◽  
Author(s):  
Nicholas C. Hunt ◽  
Andrea M. Scheetz
Keyword(s):  
Mechanical Turk ◽  
Amazon Mechanical Turk ◽  
Alternative Methods ◽  
Technical Knowledge ◽  
Data Validation ◽  
High Quality ◽  
Manipulation Check ◽  
Screening Questions ◽  
Methodological Considerations ◽  
Firsthand Experience

ABSTRACT Amazon Mechanical Turk (MTurk) is a powerful tool that is more commonly being used to recruit behavioral research participants for accounting research. This manuscript provides practical and technical knowledge learned from firsthand experience to help researchers collect high-quality, defendable data for research purposes. We highlight two issues of particular importance when using MTurk: (1) accessing qualified participants, and (2) validating collected data. To address these issues, we discuss alternative methods of carrying out screens and different data validation techniques researchers may want to consider. We also demonstrate how some of the techniques discussed were implemented for a recent data collection. Finally, we contrast the use of unpaid screens with merely putting participation requirements in the MTurk instructions to examine the effectiveness of using screens. We find that screening questions significantly reduce the number of manipulation check failures as well as significantly increase the usable responses per paid participant.

scholarly journals SOCIAL ENGAGEMENT THROUGH VIDEO CHAT FOR OLDER INDIVIDUALS WITH AND WITHOUT COGNITIVE IMPAIRMENT

2019 ◽  
Vol 3 (Supplement_1) ◽  
pp. S15-S15 ◽  
Author(s):  
Wendy Rogers ◽  
Qiong Nie ◽  
Lydia Nguyen ◽  
Raksha Mudar ◽  
Dillon Myers ◽  
...  
Keyword(s):  
Older Adults ◽  
Cognitive Impairment ◽  
Social Engagement ◽  
Older Individuals ◽  
Life Outcomes ◽  
Web Based ◽  
Video Chat ◽  
Potential Benefits ◽  
Prevailing View

Abstract Social engagement is a fundamental component of health and quality-of-life outcomes. However, there is a prevailing view that older adults primarily want to engage socially with current family and friends – that they are not interested in developing new relationships. That is an overgeneralization. We have found that older adults are interested in the opportunity to engage in social interactions with people who have shared interests. Technology can facilitate these interactions. We will describe our research with OneClick.chat, a web-based video chat system. We explored potential benefits of use by adults aged 70-85, including those with mild cognitive impairment (MCI), as well as barriers and facilitators to adoption. Participants saw value of this online social engagement platform and were able to use it with some initial training. They envisioned using OneClick not only for conversations but also for learning and doing activities with like-minded individuals.

scholarly journals Comparing Traditional and Crowdsourcing Methods for Pretesting Survey Questions

SAGE Open ◽  
2016 ◽  
Vol 6 (4) ◽  
pp. 215824401667177 ◽  
Author(s):  
Jennifer Edgar ◽  
Joe Murphy ◽  
Michael Keating
Keyword(s):  
Collective Intelligence ◽  
Amazon Mechanical Turk ◽  
Cognitive Interviewing ◽  
Cognitive Interviews ◽  
Quality Of Data ◽  
The Public ◽  
Survey Questions ◽  
Participant Characteristics ◽  
Cost Efficient

Cognitive interviewing is a common method used to evaluate survey questions. This study compares traditional cognitive interviewing methods with crowdsourcing, or “tapping into the collective intelligence of the public to complete a task.” Crowdsourcing may provide researchers with access to a diverse pool of potential participants in a very timely and cost-efficient way. Exploratory work found that crowdsourcing participants, with self-administered data collection, may be a viable alternative, or addition, to traditional pretesting methods. Using three crowdsourcing designs (TryMyUI, Amazon Mechanical Turk, and Facebook), we compared the participant characteristics, costs, and quantity and quality of data with traditional laboratory-based cognitive interviews. Results suggest that crowdsourcing and self-administered protocols may be a viable way to collect survey pretesting information, as participants were able to complete the tasks and provide useful information; however, complex tasks may require the skills of an interviewer to administer unscripted probes.

scholarly journals Fake News and Rumour Detection on Social Media

2019 ◽  
Vol 8 (11S) ◽  
pp. 1172-1175
Keyword(s):  
Mass Scale ◽  
Fake News ◽  
Quality Of Data ◽  
Mediated Communication ◽  
Web Based ◽  
Online Journals ◽  
The People ◽  
Network Propagation ◽  
The Continuum

PC Mediated Communication (CMC) advances like, for example, online journals, Twitter, Reddit, Facebook and other web based life presently have such a large number of dynamic clients that they have turned into an ideal stage for news conveyance on a mass scale. Such a mass scale news conveyance framework accompanies a proviso of faulty veracity. Building up the unwavering quality of data online is a strenuous and an overwhelming test yet it is basically essential particularly amid the time-touchy circumstances, for example, genuine crises which can have destructive impact on people and society. 2016 US Presidential race is an encapsulation of the previously mentioned crisis. In a study , it was concluded that the public's engagement with phoney news through Facebook was higher than through standard sources. So as to battle the spread of malevolent and unplanned falsehood in online networking we built up a model to recognise counterfeit news. Counterfeit news recognition is a procedure of classifying news and estimating it on the continuum of veracity. Detection is done by classifying and clustering assertions made about the event followed by veracity assessment methods emerging from linguistic cue, characteristics of the people involved and network propagation dynamics..

scholarly journals You Get What You Pay for on Health Care Question and Answer Platforms: Nonparticipant Observational Study

10.2196/13534 ◽  
2020 ◽  
Vol 22 (1) ◽  
pp. e13534
Author(s):  
Fatemeh Ameri ◽  
Kathleen Keeling ◽  
Reza Salehnejad
Keyword(s):  
Health Information ◽  
The Internet ◽  
Design Features ◽  
High Quality ◽  
Platform Design ◽  
Web Based ◽  
Health Advice ◽  
Quality Health ◽  
Advice Quality

Background Seeking health information on the internet is very popular despite the debatable ability of lay users to evaluate the quality of health information and uneven quality of information available on the Web. Consulting the internet for health information is pervasive, particularly when other sources are inaccessible because of time, distance, and money constraints or when sensitive or embarrassing questions are to be explored. Question and answer (Q&A) platforms are Web-based services that provide personalized health advice upon the information seekers’ request. However, it is not clear how the quality of health advices is ensured on these platforms. Objective The objective of this study was to identify how platform design impacts the quality of Web-based health advices and equal access to health information on the internet. Methods A total of 900 Q&As were collected from 9 Q&A platforms with different design features. Data on the design features for each platform were generated. Paid physicians evaluated the data to quantify the quality of health advices. Guided by the literature, the design features that affected information quality were identified and recorded for each Q&A platform. The least absolute shrinkage and selection operator and unbiased regression tree methods were used for the analysis. Results Q&A platform design and health advice quality were related. Expertise of information providers (beta=.48; P=.001), financial incentive (beta=.4; P=.001), external reputation (beta=.28; P=.002), and question quality (beta=.12; P=.001) best predicted health advice quality. Virtual incentive, Web 2.0 mechanisms, and reputation systems were not associated with health advice quality. Conclusions Access to high-quality health advices on the internet is unequal and skewed toward high-income and high-literacy groups. However, there are possibilities to generate high-quality health advices for free.

Improving consistency of high quality diagnosis, staging, and treatment using measurement.

2017 ◽  
Vol 35 (8_suppl) ◽  
pp. 217-217
Author(s):  
Shaheena Mukhi ◽  
John Srigley ◽  
Corinne Daly ◽  
Mary Agent-Katwala
Keyword(s):  
Lymph Node ◽  
Quality Of Data ◽  
Group Discussions ◽  
High Quality ◽  
Multidisciplinary Group ◽  
Canadian Provinces ◽  
Node Removal ◽  
Quality Diagnosis ◽  
Culture Of Quality

217 Background: To improve variability in diagnosing and treating cancer resection cases, six Canadian provinces implemented standardized pathology checklists to transition from narrative to synoptic reporting. In clinical practice, pathologists are electronically capturing data on the resected cancer specimens synoptically for breast, colorectal, lung, prostate, and endometrial cases. Though data were collected in a standardized format, consensus based indicators were unavailable to coordinate action across Canada. Objectives: We aimed to develop indicators to measure consistency of high quality cancer diagnosis, staging, prognosis and treatment, and coordinate action. Methods: A literature review was conducted with the input of clinical experts to inform the development of indicators. 50 clinicians from x jurisdictions reviewed, selected and ranked 33 indicators, initially drafted. Clinicians also provided input on the clinical validity of the indicators and set targets based on evidence. Clinicians reviewed the baseline data, confirmed the clinical usefulness of indicators, and assigned indicators into three pioneered domains. Results: 47 indicators were developed and categorized into one of three domains: descriptive, which provide data on intrinsic measures of a patient’s tumour, such as stage or tumour type; process, which measure the quality of data completeness, timeliness and compliance; and clinico-pathologic outcome, which examine surgeon or pathologist effect on the diagnostic pathway, such as margin positivity rates or adequacy of lymph node removal. Examples of indicators are: margin status; lymph node examined, involved and retrieval; histologic type and grade distribution; lympho-vascular invasion; pT3 margin positivity rate. Conclusions: The indicators have set a framework for: measuring consistency and inconsistency in diagnosing and staging cancer; for organizing conversations and multidisciplinary group discussions; and establishing the culture of quality improvement.

Sign in / Sign up

Export Citation Format

Share Document