Online Controlled Experiments at Large Scale in Society 5.0

Author(s):  
Ron Kohavi
Author(s):  
Xin (Shane) Wang ◽  
Shijie Lu ◽  
X I Li ◽  
Mansur Khamitov ◽  
Neil Bendle

Abstract Persuasion success is often related to hard-to-measure characteristics, such as the way the persuader speaks. To examine how vocal tones impact persuasion in an online appeal, this research measures persuaders’ vocal tones in Kickstarter video pitches using novel audio mining technology. Connecting vocal tone dimensions with real-world funding outcomes offers insight into the impact of vocal tones on receivers’ actions. The core hypothesis of this paper is that a successful persuasion attempt is associated with vocal tones denoting (1) focus, (2) low stress, and (3) stable emotions. These three vocal tone dimensions—which are in line with the stereotype content model—matter because they allow receivers to make inferences about a persuader’s competence. The hypotheses are tested with a large-scale empirical study using Kickstarter data, which is then replicated in a different category. In addition, two controlled experiments provide evidence that perceptions of competence mediate the impact of the three vocal tones on persuasion attempt success. The results identify key indicators of persuasion attempt success and suggest a greater role for audio mining in academic consumer research.


2017 ◽  
Author(s):  
Alexander Olof Savi ◽  
Joseph Jay Williams ◽  
Gunter Maris ◽  
Han van der Maas

Although large-scale online learning increasingly succeeds in attracting learners worldwide, to date it fails to deliver on its promise. We first show the immense popularity of online learning and discuss its (unsatisfactory) effectiveness. We then discuss large-scale online randomized controlled experiments (A/B tests) as a powerful complimentary means to enable the desired leap forward. Although these experiments are widely and intensively used for web page optimization, and are slowly being adopted by the online learning community, their use, benefits, and challenges have only limitedly seeped through to the larger learning community. We summarize existing efforts in employing A/B tests in online learning, argue that such tests should take into account the typical nature of (online) learning, and encourage the use of knowledge from the various learning sciences to identify interventions that promise improved learning. We finally discuss both the limitations and promises of A/B tests, and show how such tests can ultimately contribute to learning that is tailored to each individual learner. The insights and priorities that arise from this overview and synthesis of A/B tests in online learning may help advance and direct the field.


2021 ◽  
Vol 9 ◽  
pp. 978-994
Author(s):  
Emanuele Bugliarello ◽  
Ryan Cotterell ◽  
Naoaki Okazaki ◽  
Desmond Elliott

Abstract Large-scale pretraining and task-specific fine- tuning is now the standard methodology for many tasks in computer vision and natural language processing. Recently, a multitude of methods have been proposed for pretraining vision and language BERTs to tackle challenges at the intersection of these two key areas of AI. These models can be categorized into either single-stream or dual-stream encoders. We study the differences between these two categories, and show how they can be unified under a single theoretical framework. We then conduct controlled experiments to discern the empirical differences between five vision and language BERTs. Our experiments show that training data and hyperparameters are responsible for most of the differences between the reported results, but they also reveal that the embedding layer plays a crucial role in these massive models.


2017 ◽  
Vol 119 (3) ◽  
pp. 1-36 ◽  
Author(s):  
Korinn Ostrow ◽  
Neil Heffernan ◽  
Joseph Jay Williams

Background/Context Large-scale randomized controlled experiments conducted in authentic learning environments are commonly high stakes, carrying extensive costs and requiring lengthy commitments for all-or-nothing results amidst many potential obstacles. Educational technologies harbor an untapped potential to provide researchers with access to extensive and diverse subject pools of students interacting with educational materials in authentic ways. These systems log extensive data on student performance that can be used to identify and leverage best practices in education and guide systemic policy change. Tomorrow's educational technologies should be built upon rigorous standards set forth by the research revolution budding today. Purpose/Objective/Research Question/Focus of Study The present work serves as a call to the community to infuse popular learning platforms with the capacity to support collaborative research at scale. Research Design This article defines how educational technologies can be leveraged for use in collaborative research environments by highlighting the research revolution of ASSISTments ( www.ASSISTments.org ), a popular online learning platform with a focus on mathematics education. A framework described as the cycle of perpetual evolution is presented, and research exemplifying progression through this framework is discussed in support of the many benefits that stem from infusing EdTech with collaborative research. Through a recent NSF grant (SI2-SSE&SSI: 1440753), researchers from around the world can leverage ASSISTments’ content and user population by designing and implementing randomized controlled experiments within the ASSISTments TestBed ( www.ASSISTmentsTestBed.org ). Findings from these studies help to define best practices within technology-driven learning, while simultaneously allowing for augmentation of the system's content, delivery, and infrastructure. Conclusions/Recommendations Supplementing educational technologies with environments for sound, collaborative science can result in a broad range of benefits for students, researchers, platforms, and educational practice and policy. This article outlines the successful uptake of research efforts by ASSISTments in hopes of advocating a research revolution for other educational technologies.


Author(s):  
Ron Kohavi ◽  
Alex Deng ◽  
Brian Frasca ◽  
Toby Walker ◽  
Ya Xu ◽  
...  

2000 ◽  
Vol 46 (2) ◽  
pp. 233-251 ◽  
Author(s):  
Martin Killias ◽  
Marcelo F. Aebi ◽  
Denis Ribeaud

Europe, over the past two decades, has seen many innovations in the field of corrections, particularly new sanctions that are becoming increasingly popular as alternatives to imprisonment, such as community service. Innovative approaches have also been tested in the field of drug treatment, including large-scale drug-substitution programs. Usually, such programs have been evaluated, if at all, under the form of before-after studies. Thus, little is known about treatment effects, particularly in the longer run and/or compared to alternative approaches. Two controlled experiments conducted recently in Switzerland involving community service and heroin prescription to addicts may indicate a shift to more rigorous evaluations. They both illustrate the potentials of controlled experiments for progress in knowledge as well as some problems in methodological, legal, ethical, and practical respects. Whereas controlled experiments are necessary to learn in some areas, more conventional before-after studies may be valid under particular circumstances.


2020 ◽  
Vol 117 (37) ◽  
pp. 22787-22792 ◽  
Author(s):  
Alexander Ehlert ◽  
Martin Kindschi ◽  
René Algesheimer ◽  
Heiko Rauhut

While it is undeniable that the ability of humans to cooperate in large-scale societies is unique in animal life, it remains open how such a degree of prosociality is possible despite the risks of exploitation. Recent evidence suggests that social networks play a crucial role in the development of prosociality and large-scale cooperation by allowing cooperators to cluster; however, it is not well understood if and how this also applies to real-world social networks in the field. We study intrinsic social preferences alongside emerging friendship patterns in 57 freshly formed school classes (n = 1,217), using incentivized measures. We demonstrate the existence of cooperative clusters in society, examine their emergence, and expand the evidence from controlled experiments to real-world social networks. Our results suggest that being embedded in cooperative environments substantially enhances the social preferences of individuals, thus contributing to the formation of cooperative clusters. Partner choice, in contrast, only marginally contributes to their emergence. We conclude that cooperative preferences are contagious; social and cultural learning plays an important role in the development and evolution of cooperation.


1999 ◽  
Vol 173 ◽  
pp. 243-248
Author(s):  
D. Kubáček ◽  
A. Galád ◽  
A. Pravda

AbstractUnusual short-period comet 29P/Schwassmann-Wachmann 1 inspired many observers to explain its unpredictable outbursts. In this paper large scale structures and features from the inner part of the coma in time periods around outbursts are studied. CCD images were taken at Whipple Observatory, Mt. Hopkins, in 1989 and at Astronomical Observatory, Modra, from 1995 to 1998. Photographic plates of the comet were taken at Harvard College Observatory, Oak Ridge, from 1974 to 1982. The latter were digitized at first to apply the same techniques of image processing for optimizing the visibility of features in the coma during outbursts. Outbursts and coma structures show various shapes.


1994 ◽  
Vol 144 ◽  
pp. 29-33
Author(s):  
P. Ambrož

AbstractThe large-scale coronal structures observed during the sporadically visible solar eclipses were compared with the numerically extrapolated field-line structures of coronal magnetic field. A characteristic relationship between the observed structures of coronal plasma and the magnetic field line configurations was determined. The long-term evolution of large scale coronal structures inferred from photospheric magnetic observations in the course of 11- and 22-year solar cycles is described.Some known parameters, such as the source surface radius, or coronal rotation rate are discussed and actually interpreted. A relation between the large-scale photospheric magnetic field evolution and the coronal structure rearrangement is demonstrated.


Sign in / Sign up

Export Citation Format

Share Document