scholarly journals Deep exploration as a unifying account of explore-exploit behavior

2020 ◽  
Author(s):  
Robert C Wilson ◽  
Siyu Wang ◽  
Hashem Sadeghiyeh ◽  
Jonathan D. Cohen
Keyword(s):  
Machine Learning ◽  
Decision Making ◽  
Human Behavior ◽  
Information Seeking ◽  
The Other ◽  
Behavioral Variability ◽  
Simple Task ◽  
Stochastic Planning ◽  
Human Decision ◽  
Deep Exploration

Many decisions involve a choice between exploring unknown opportunities and exploiting well-known options. Work across a variety of domains, from animal foraging to human decision making, has suggested that animals solve such ``explore-exploit dilemmas'' with a mixture of two strategies: one driven by information seeking (directed exploration) and the other by behavioral variability (random exploration). Here we propose a unifying account in which these two strategies emerge from a kind of stochastic planning, known in the machine learning literature as Deep Exploration. In this model, the explore-exploit decision is made by stochastic simulation of plausible futures that are deep, in that they extend far into the future, and narrow, in that the number of possible futures they consider is small. By applying Deep Exploration to a simple explore-exploit task we show theoretically how directed and random exploration can emerge in these settings. Moreover, we show that Deep Exploration implies a tradeoff between directed and random exploration that is mediated by the number of simulations, or samples --- with more samples leading to increased directed exploration and decreased random exploration at the expense of greater time taken to respond. By measuring human behavior on the same simple task, we show that this reaction-time-mediated tradeoff exists in human behavior both between and within participants. We therefore suggest that Deep Exploration is a unifying account of explore-exploit behavior in humans.

scholarly journals Any way the brain blows? The nature of decision noise in random exploration

2018 ◽  
Author(s):  
Siyu Wang ◽  
Robert C Wilson
Keyword(s):  
Decision Making ◽  
Cognitive Control ◽  
Recent Work ◽  
Human Behavior ◽  
External Noise ◽  
Behavioral Variability ◽  
The World ◽  
Human Decision ◽  
Adaptive Noise ◽  
The Brain

Human decision making is inherently variable. While this variability is often seen as a sign of suboptimality in human behavior, recent work suggests that randomness can actually be adaptive. An example arises when we must choose between exploring unknown options or exploiting options we know well. A little randomness in these `explore-exploit' decisions is remarkably effective as it encourages us to explore options we might otherwise ignore. Moreover, people actually use such `random exploration' in practice, increasing their behavioral variability when it is more valuable to explore. Despite this progress, the nature of adaptive `decision noise' for exploration is unknown -- specifically whether it is generated internally, from stochastic processes in the brain, or externally, from stochastic stimuli in the world. Here we show that, while both internal and external noise drive variability in behavior, the noise driving random exploration is predominantly internal. This suggests that random exploration depends on adaptive noise processes in the brain which are subject to cognitive control.

Who’s making the decisions? How managers can harness artificial intelligence and remain in charge

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Pooya Tabesh
Keyword(s):  
Artificial Intelligence ◽  
Machine Learning ◽  
Decision Making ◽  
Big Data ◽  
Digital Transformation ◽  
Decision Processes ◽  
Decision Makers ◽  
Content Type ◽  
Human Decision ◽  
Practical Implications

Purpose While it is evident that the introduction of machine learning and the availability of big data have revolutionized various organizational operations and processes, existing academic and practitioner research within decision process literature has mostly ignored the nuances of these influences on human decision-making. Building on existing research in this area, this paper aims to define these concepts from a decision-making perspective and elaborates on the influences of these emerging technologies on human analytical and intuitive decision-making processes. Design/methodology/approach The authors first provide a holistic understanding of important drivers of digital transformation. The authors then conceptualize the impact that analytics tools built on artificial intelligence (AI) and big data have on intuitive and analytical human decision processes in organizations. Findings The authors discuss similarities and differences between machine learning and two human decision processes, namely, analysis and intuition. While it is difficult to jump to any conclusions about the future of machine learning, human decision-makers seem to continue to monopolize the majority of intuitive decision tasks, which will help them keep the upper hand (vis-à-vis machines), at least in the near future. Research limitations/implications The work contributes to research on rational (analytical) and intuitive processes of decision-making at the individual, group and organization levels by theorizing about the way these processes are influenced by advanced AI algorithms such as machine learning. Practical implications Decisions are building blocks of organizational success. Therefore, a better understanding of the way human decision processes can be impacted by advanced technologies will prepare managers to better use these technologies and make better decisions. By clarifying the boundaries/overlaps among concepts such as AI, machine learning and big data, the authors contribute to their successful adoption by business practitioners. Social implications The work suggests that human decision-makers will not be replaced by machines if they continue to invest in what they do best: critical thinking, intuitive analysis and creative problem-solving. Originality/value The work elaborates on important drivers of digital transformation from a decision-making perspective and discusses their practical implications for managers.

scholarly journals A journey of a thousand miles begins with one small step – human agency, hydrological processes and time in socio-hydrology

2013 ◽  
Vol 10 (11) ◽  
pp. 14265-14304 ◽  
Author(s):  
M. W. Ertsen ◽  
J. T. Murphy ◽  
L. E. Purdue ◽  
T. Zhu
Keyword(s):  
Decision Making ◽  
Social Action ◽  
Human Agency ◽  
The Other ◽  
Social Structures ◽  
Small Scale ◽  
Small Step ◽  
Time Step ◽  
Human Decision ◽  
Day To Day Operations

Abstract. When simulating social action in modeling efforts, as in socio-hydrology, an issue of obvious importance is how to ensure that social action by human agents is well-represented in the analysis and the model. Generally, human decision-making is either modeled on a yearly basis or lumped together as collective social structures. Both responses are problematic, as human decision making is more complex and organizations are the result of human agency and cannot be used as explanatory forces. A way out of the dilemma how to include human agency is to go to the largest societal and environmental clustering possible: society itself and climate, with time steps of years or decades. In the paper, the other way out is developed: to face human agency squarely, and direct the modeling approach to the human agency of individuals and couple this with the lowest appropriate hydrological level and time step. This approach is supported theoretically by the work of Bruno Latour, the French sociologist and philosopher. We discuss irrigation archaeology, as it is in this discipline that the issues of scale and explanatory force are well discussed. The issue is not just what scale to use: it is what scale matters. We argue that understanding the arrangements that permitted the management of irrigation over centuries, requires modeling and understanding the small-scale, day-to-day operations and personal interactions upon which they were built. This effort, however, must be informed by the longer-term dynamics as these provide the context within which human agency, is acted out.

Integration of Data Mining and Operations Research

2011 ◽  
pp. 1046-1052
Author(s):  
Stephan Meisel
Keyword(s):  
Machine Learning ◽  
Data Mining ◽  
Decision Making ◽  
Operations Research ◽  
Secondary Analysis ◽  
The Other ◽  
Optimization Models ◽  
Optimal Solutions ◽  
Number Of Publications ◽  
The One

Basically, Data Mining (DM) and Operations Research (OR) are two paradigms independent of each other. OR aims at optimal solutions of decision problems with respect to a given goal. DM is concerned with secondary analysis of large amounts of data (Hand et al., 2001). However, there are some commonalities. Both paradigms are application focused (Wu et al., 2003; White, 1991). Many Data Mining approaches are within traditional OR domains like logistics, manufacturing, health care or finance. Further, both DM and OR are multidisciplinary. Since its origins, OR has been relying on fields such as mathematics, statistics, economics and computer science. In DM, most of the current textbooks show a strong bias towards one of its founding disciplines, like database management, machine learning or statistics. Being multidisciplinary and application focused, it seems to be a natural step for both paradigms to gain synergies from integration. Thus, recently an increasing number of publications of successful approaches at the intersection of DM and OR can be observed. On the one hand, efficiency of the DM process is increased by use of advanced optimization models and methods originating from OR. On the other hand, effectiveness of decision making is increased by augmentation of traditional OR approaches with DM results. Meisel and Mattfeld (in press) provide a detailed discussion of the synergies of DM and OR.

Robot Process Automation (RPA) and Its Future

2022 ◽  
pp. 35-58
Author(s):  
Ozge Doguc
Keyword(s):  
Artificial Intelligence ◽  
Machine Learning ◽  
Computer Vision ◽  
Customer Satisfaction ◽  
Human Behavior ◽  
The Other ◽  
Human Intervention ◽  
Process Automation ◽  
Robotic Process Automation ◽  
Human Agent

Many software automation techniques have been developed in the last decade to cut down cost, improve customer satisfaction, and reduce errors. Robotic process automation (RPA) has become increasingly popular recently. RPA offers software robots (bots) that can mimic human behavior. Attended robots work in tandem with humans and can operate while the human agent is active on the computer. On the other hand, unattended robots operate behind locked screens and are designed to execute automations that don't require any human intervention. RPA robots are equipped with artificial intelligence engines such as computer vision and machine learning, and both robot types can learn automations by recording human actions.

Editors’ Introduction

2017 ◽  
Author(s):  
Wim Bernasco ◽  
Henk Elffers ◽  
Jean-Louis van Gelder
Keyword(s):  
Decision Making ◽  
Law Enforcement ◽  
Empirical Evidence ◽  
Research Methods ◽  
Human Behavior ◽  
Human Decision ◽  
Criminal Conduct

Decision making is central to all human behavior, including criminal conduct. Virtually every discussion about crime or law enforcement is guided by beliefs about how people make decisions in one way or another. This interdisciplinary handbook integrates insights about the role of human decision making as it relates to crime. It contains reviews of the main theories of offender decision making and also reviews of empirical evidence on topics as diverse as desistance, crime locations, co-offending, victimization, and criminal methods and tools. It further includes in-depth treatments of the principal research methods for studying offender decision making and a series of chapters on specific types of crime.

Using large-scale experiments and machine learning to discover theories of human decision-making

Science ◽  
2021 ◽  
Vol 372 (6547) ◽  
pp. 1209-1214
Author(s):  
Joshua C. Peterson ◽  
David D. Bourgin ◽  
Mayank Agrawal ◽  
Daniel Reichman ◽  
Thomas L. Griffiths
Keyword(s):  
Machine Learning ◽  
Decision Making ◽  
Large Scale ◽  
Risky Choice ◽  
Machine Learning Algorithms ◽  
Psychological Theories ◽  
The Social ◽  
Power Machine ◽  
Human Decision ◽  
Gradient Based

Predicting and understanding how people make decisions has been a long-standing goal in many fields, with quantitative models of human decision-making informing research in both the social sciences and engineering. We show how progress toward this goal can be accelerated by using large datasets to power machine-learning algorithms that are constrained to produce interpretable psychological theories. Conducting the largest experiment on risky choice to date and analyzing the results using gradient-based optimization of differentiable decision theories implemented through artificial neural networks, we were able to recapitulate historical discoveries, establish that there is room to improve on existing theories, and discover a new, more accurate model of human decision-making in a form that preserves the insights from centuries of research.

Robot Process Automation (RPA) and Its Future

2020 ◽  
pp. 469-492
Author(s):  
Ozge Doguc
Keyword(s):  
Artificial Intelligence ◽  
Machine Learning ◽  
Computer Vision ◽  
Customer Satisfaction ◽  
Human Behavior ◽  
The Other ◽  
Human Intervention ◽  
Process Automation ◽  
Robotic Process Automation ◽  
Human Agent

Many software automation techniques have been developed in the last decade to cut down cost, improve customer satisfaction, and reduce errors. Robotic process automation (RPA) has become increasingly popular recently. RPA offers software robots (bots) that can mimic human behavior. Attended robots work in tandem with humans and can operate while the human agent is active on the computer. On the other hand, unattended robots operate behind locked screens and are designed to execute automations that don't require any human intervention. RPA robots are equipped with artificial intelligence engines such as computer vision and machine learning, and both robot types can learn automations by recording human actions.

MAITS

2011 ◽  
pp. 189-208 ◽  
Author(s):  
Dharmendra Sharma
Keyword(s):  
Machine Learning ◽  
Decision Making ◽  
Learning Algorithm ◽  
Computer Forensics ◽  
The Other ◽  
Security Requirements ◽  
Machine Learning Algorithm ◽  
Agent Based ◽  
Authentication And Authorization ◽  
Multi Agent

In this chapter, we propose a multi-agent-based information technology (IT) security approach (MAITS) as a holistic solution to the increasing needs of securing computer systems. Each specialist task for security requirements is modeled as a specialist agent. MAITS has five groups of working agents—administration assistant agents, authentication and authorization agents, system log *monitoring agents, intrusion detection agents, and pre-mortem-based computer forensics agents. An assessment center, which is comprised of yet another special group of agents, plays a key role in coordinating the interaction of the other agents. Each agent has an agent engine of an appropriate machine-learning algorithm. The engine enables the agent with learning, reasoning, and decision-making abilities. Each agent also has an agent interface, through which the agent interacts with other agents and also the environment.

On Predicting Recidivism: Epistemic Risk, Tradeoffs, and Values in Machine Learning

2020 ◽  
pp. 1-21
Author(s):  
Justin B. Biddle
Keyword(s):  
Machine Learning ◽  
Decision Making ◽  
Value Judgments ◽  
Epistemic Risk ◽  
Prediction Algorithms ◽  
Recent Scholarship ◽  
Recidivism Prediction ◽  
Human Decision ◽  
Technological Decision Making

Abstract Recent scholarship in philosophy of science and technology has shown that scientific and technological decision making are laden with values, including values of a social, political, and/or ethical character. This paper examines the role of value judgments in the design of machine-learning (ML) systems generally and in recidivism-prediction algorithms specifically. Drawing on work on inductive and epistemic risk, the paper argues that ML systems are value laden in ways similar to human decision making, because the development and design of ML systems requires human decisions that involve tradeoffs that reflect values. In many cases, these decisions have significant—and, in some cases, disparate—downstream impacts on human lives. After examining an influential court decision regarding the use of proprietary recidivism-prediction algorithms in criminal sentencing, Wisconsin v. Loomis, the paper provides three recommendations for the use of ML in penal systems.

Sign in / Sign up

Export Citation Format

Share Document