scholarly journals Simulating bout-and-pause patterns with reinforcement learning

PLoS ONE ◽  
2020 ◽  
Vol 15 (11) ◽  
pp. e0242201
Author(s):  
Kota Yamada ◽  
Atsunori Kanemura
Keyword(s):  
Reinforcement Learning ◽  
Computational Models ◽  
Response Rate ◽  
Temporal Structure ◽  
Initiation Rate ◽  
Proposed Model ◽  
Simulation Results ◽  
Underlying Mechanisms ◽  
Experimental Findings ◽  
Three Components

Animal responses occur according to a specific temporal structure composed of two states, where a bout is followed by a long pause until the next bout. Such a bout-and-pause pattern has three components: the bout length, the within-bout response rate, and the bout initiation rate. Previous studies have investigated how these three components are affected by experimental manipulations. However, it remains unknown what underlying mechanisms cause bout-and-pause patterns. In this article, we propose two mechanisms and examine computational models developed based on reinforcement learning. The model is characterized by two mechanisms. The first mechanism is choice—an agent makes a choice between operant and other behaviors. The second mechanism is cost—a cost is associated with the changeover of behaviors. These two mechanisms are extracted from past experimental findings. Simulation results suggested that both the choice and cost mechanisms are required to generate bout-and-pause patterns and if either of them is knocked out, the model does not generate bout-and-pause patterns. We further analyzed the proposed model and found that it reproduced the relationships between experimental manipulations and the three components that have been reported by previous studies. In addition, we showed alternative models can generate bout-and-pause patterns as long as they implement the two mechanisms.

scholarly journals Simulating bout-and-pause patterns with reinforcement learning

2019 ◽  
Author(s):  
Kota Yamada ◽  
Atsunori Kanemura
Keyword(s):  
Reinforcement Learning ◽  
Computational Models ◽  
Response Rate ◽  
Temporal Structure ◽  
Initiation Rate ◽  
Proposed Model ◽  
Simulation Results ◽  
Underlying Mechanisms ◽  
Experimental Findings ◽  
Three Components

AbstractAnimal responses occur according to a specific temporal structure composed of two states, where a bout is followed by a long pause until the next bout. Such about-and-pause pattern has three components: the bout length, the within-bout response rate, and the bout initiation rate. Previous studies have investigated how these three components are affected by experimental manipulations. However, it remains unknown what underlying mechanisms cause bout-and-pause patterns. In this article, we propose two mechanisms and examine computational models developed based on reinforcement learning. The model is characterized by two mechanisms. The first mechanism is choice—an agent makes a choice between operant and other behaviors. The second mechanism is cost—a cost is associated with the changeover of behaviors. These two mechanisms are extracted from past experimental findings. Simulation results suggested that both the choice and cost mechanisms are required to generate bout-and-pause patterns and if either of them is knocked out, the model does not generate bout-and-pause patterns. We further analyzed the proposed model and found that it reproduced the relationships between experimental manipulations and the three components that have been reported by previous studies. In addition, we showed alternative models can generate bout-and-pause patterns as long as they implement the two mechanisms.

scholarly journals SARL: A reinforcement learning based QoS-aware IoT service discovery model

2020 ◽  
Vol 71 (6) ◽  
pp. 368-378
Author(s):  
Selahattin Kosunalp ◽  
Kubilay Demir
Keyword(s):  
Reinforcement Learning ◽  
Service Discovery ◽  
Discovery Process ◽  
Energy Usage ◽  
Proposed Model ◽  
Enormous Amount ◽  
Simulation Results ◽  
Composition Process ◽  
Message Overhead ◽  
Usage Efficiency

AbstractThe IoT environment includes the enormous amount of atomic services with dynamic QoS compared with traditional web services. In such an environment, in the service composition process, discovering a requested service meeting the required QoS is a di cult task. In this work, to address this issue, we propose a peer-to-peer-based service discovery model, which looks for the information about services meeting the requested QoS and functionality on an overlay constructed with users of services versus service nodes, with probably constrained resources. However, employing a plain discovery algorithm on the overlay network such as flooding, or k-random walk could cause high message overhead or delay. This necessitates an intelligent and adaptive discovery algorithm, which adapts itself based on users’ previous queries and the results. To fill this gap, the proposed service discovery approach is equipped with a reinforcement learning-based algorithm, named SARL. The reinforcement learning-based algorithm enables SARL to significantly reduce delay and message overhead in the service discovery process by ranking neighboring nodes based on users’ service request preferences and the service query results. The proposed model is implemented on the OMNet simulation platform. The simulation results demonstrate that SARL remarkably outperforms the existing approaches in terms of message overhead, reliability, timeliness, and energy usage efficiency.

Soil Moisture based Automatic Irrigation System to Improve Water Productivity

2020 ◽  
Vol 7 (04) ◽  
Author(s):  
PRADEEP H K ◽  
JASMA BALASANGAMESHWARA ◽  
K RAJAN ◽  
PRABHUDEV JAGADEESH
Keyword(s):  
Soil Moisture ◽  
Soil Moisture Content ◽  
Irrigation System ◽  
Water Productivity ◽  
Vital Role ◽  
Agricultural Water Management ◽  
Water Management System ◽  
Proposed Model ◽  
Crop Growth Stage ◽  
Experimental Findings

Irrigation automation plays a vital role in agricultural water management system. An efficient automatic irrigation system is crucial to improve crop water productivity. Soil moisture based irrigation is an economical and efficient approach for automation of irrigation system. An experiment was conducted for irrigation automation based on the soil moisture content and crop growth stage. The experimental findings exhibited that, automatic irrigation system based on the proposed model triggers the water supply accurately based on the real-time soil moisture values.

scholarly journals On the structural connectivity of large-scale models of brain networks at cellular level

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Giuseppe Giacopelli ◽  
Domenico Tegolo ◽  
Emiliano Spera ◽  
Michele Migliore
Keyword(s):  
Large Scale ◽  
Brain Networks ◽  
Structural Connectivity ◽  
Network Models ◽  
Brain Regions ◽  
Cellular Level ◽  
Scale Model ◽  
Mathematical Framework ◽  
Underlying Mechanisms ◽  
Experimental Findings

AbstractThe brain’s structural connectivity plays a fundamental role in determining how neuron networks generate, process, and transfer information within and between brain regions. The underlying mechanisms are extremely difficult to study experimentally and, in many cases, large-scale model networks are of great help. However, the implementation of these models relies on experimental findings that are often sparse and limited. Their predicting power ultimately depends on how closely a model’s connectivity represents the real system. Here we argue that the data-driven probabilistic rules, widely used to build neuronal network models, may not be appropriate to represent the dynamics of the corresponding biological system. To solve this problem, we propose to use a new mathematical framework able to use sparse and limited experimental data to quantitatively reproduce the structural connectivity of biological brain networks at cellular level.

Wear of a Tool in Double-Disk Lapping of Silicon Wafers

2010 ◽  
Author(s):  
Adam Barylski ◽  
Mariusz Deja
Keyword(s):  
Experimental Data ◽  
Integrated Circuits ◽  
Tool Wear ◽  
Silicon Wafers ◽  
Silicon Ingot ◽  
Proposed Model ◽  
Tool Wear Prediction ◽  
Simulation Results ◽  
High Degree ◽  
Kinematical Parameters

Silicon wafers are the most widely used substrates for fabricating integrated circuits. A sequence of processes is needed to turn a silicon ingot into silicon wafers. One of the processes is flattening by lapping or by grinding to achieve a high degree of flatness and parallelism of the wafer [1, 2, 3]. Lapping can effectively remove or reduce the waviness induced by preceding operations [2, 4]. The main aim of this paper is to compare the simulation results with lapping experimental data obtained from the Polish producer of silicon wafers, the company Cemat Silicon from Warsaw (www.cematsil.com). Proposed model is going to be implemented by this company for the tool wear prediction. Proposed model can be applied for lapping or grinding with single or double-disc lapping kinematics [5, 6, 7]. Geometrical and kinematical relations with the simulations are presented in the work. Generated results for given workpiece diameter and for different kinematical parameters are studied using models programmed in the Matlab environment.

On the Utility of Crystal Plasticity Modeling to Uncover the Individual Roles of Microdeformation Mechanisms on the Work Hardening Response of Fe-23Mn-0.5C TWIP Steel in the Presence of Hydrogen

2018 ◽  
Vol 140 (3) ◽  
Author(s):  
B. Bal ◽  
M. Koyama ◽  
D. Canadinc ◽  
G. Gerstein ◽  
H. J. Maier ◽  
...  
Keyword(s):  
Electron Microscopy ◽  
Hydrogen Embrittlement ◽  
Twip Steel ◽  
Strain Response ◽  
Deformation Response ◽  
Self Consistent ◽  
Twip Steels ◽  
Simulation Results ◽  
The Individual ◽  
Experimental Findings

This paper presents a combined experimental and theoretical analysis focusing on the individual roles of microdeformation mechanisms that are simultaneously active during the deformation of twinning-induced plasticity (TWIP) steels in the presence of hydrogen. Deformation responses of hydrogen-free and hydrogen-charged TWIP steels were examined with the aid of thorough electron microscopy. Specifically, hydrogen charging promoted twinning over slip–twin interactions and reduced ductility. Based on the experimental findings, a mechanism-based microscale fracture model was proposed, and incorporated into a visco-plastic self-consistent (VPSC) model to account for the stress–strain response in the presence of hydrogen. In addition, slip-twin and slip–grain boundary interactions in TWIP steels were also incorporated into VPSC, in order to capture the deformation response of the material in the presence of hydrogen. The simulation results not only verify the success of the proposed hydrogen embrittlement (HE) mechanism for TWIP steels, but also open a venue for the utility of these superior materials in the presence of hydrogen.

Methods of Mathematical Modeling of the Leaching Process of Cobalt-Containing Solutions

2021 ◽  
Vol 316 ◽  
pp. 661-666
Author(s):  
Nataliya V. Mokrova
Keyword(s):  
Mathematical Modeling ◽  
Mathematical Model ◽  
Chemical Kinetics ◽  
Group Method ◽  
Data Handling ◽  
Multistage Processes ◽  
Leaching Process ◽  
Proposed Model ◽  
Simulation Results ◽  
The Mathematical Model

Current cobalt processing practices are described. This article discusses the advantages of the group argument accounting method for mathematical modeling of the leaching process of cobalt solutions. Identification of the mathematical model of the cascade of reactors of cobalt-producing is presented. Group method of data handling is allowing: to eliminate the need to calculate quantities of chemical kinetics; to get the opportunity to take into account the results of mixed experiments; to exclude the influence of random interference on the simulation results. The proposed model confirms the capabilities of the group method of data handling for describing multistage processes.

Collaborative Scheduling of Algorithms for Path Planning of Unmanned Systems

2021 ◽  
Vol 01 ◽  
Author(s):  
Ying Li ◽  
Chubing Guo ◽  
Jianshe Wu ◽  
Xin Zhang ◽  
Jian Gao ◽  
...  
Keyword(s):  
Genetic Algorithm ◽  
Reinforcement Learning ◽  
Path Planning ◽  
Unmanned Systems ◽  
Ant Colony Optimization Algorithm ◽  
A Algorithm ◽  
Collaborative Scheduling ◽  
Simulation Results ◽  
Planning Problems ◽  
Effective Path

Background: Unmanned systems have been widely used in multiple fields. Many algorithms have been proposed to solve path planning problems. Each algorithm has its advantages and defects and cannot adapt to all kinds of requirements. An appropriate path planning method is needed for various applications. Objective: To select an appropriate algorithm fastly in a given application. This could be helpful for improving the efficiency of path planning for Unmanned systems. Methods: This paper proposes to represent and quantify the features of algorithms based on the physical indicators of results. At the same time, an algorithmic collaborative scheme is developed to search the appropriate algorithm according to the requirement of the application. As an illustration of the scheme, four algorithms, including the A-star (A*) algorithm, reinforcement learning, genetic algorithm, and ant colony optimization algorithm, are implemented in the representation of their features. Results: In different simulations, the algorithmic collaborative scheme can select an appropriate algorithm in a given application based on the representation of algorithms. And the algorithm could plan a feasible and effective path. Conclusion: An algorithmic collaborative scheme is proposed, which is based on the representation of algorithms and requirement of the application. The simulation results prove the feasibility of the scheme and the representation of algorithms.

scholarly journals A strategy learning model for autonomous agents based on classification

2015 ◽  
Vol 25 (3) ◽  
pp. 471-482 ◽  
Author(s):  
Bartłomiej Śnieżyński
Keyword(s):  
Reinforcement Learning ◽  
Supervised Learning ◽  
Learning Process ◽  
Autonomous Agents ◽  
Good Alternative ◽  
Learning Model ◽  
Learning Method ◽  
Complex Environments ◽  
Agent Based ◽  
Proposed Model

AbstractIn this paper we propose a strategy learning model for autonomous agents based on classification. In the literature, the most commonly used learning method in agent-based systems is reinforcement learning. In our opinion, classification can be considered a good alternative. This type of supervised learning can be used to generate a classifier that allows the agent to choose an appropriate action for execution. Experimental results show that this model can be successfully applied for strategy generation even if rewards are delayed. We compare the efficiency of the proposed model and reinforcement learning using the farmer-pest domain and configurations of various complexity. In complex environments, supervised learning can improve the performance of agents much faster that reinforcement learning. If an appropriate knowledge representation is used, the learned knowledge may be analyzed by humans, which allows tracking the learning process

ACCURATE ANALYSIS OF GLOBAL INTERCONNECTS IN NANO-FPGAs

NANO ◽  
2009 ◽  
Vol 04 (03) ◽  
pp. 171-176 ◽  
Author(s):  
DAVOOD FATHI ◽  
BEHJAT FOROUZANDEH
Keyword(s):  
Propagation Delay ◽  
Modeling Method ◽  
Accurate Analysis ◽  
Proposed Model ◽  
Simulation Results ◽  
The Difference ◽  
A New Technique ◽  
Global Interconnects ◽  
Technology Nodes ◽  
Model Technique

This paper introduces a new technique for analyzing the behavior of global interconnects in FPGAs, for nanoscale technologies. Using this new enhanced modeling method, new enhanced accurate expressions for calculating the propagation delay of global interconnects in nano-FPGAs have been derived. In order to verify the proposed model, we have performed the delay simulations in 45 nm, 65 nm, 90 nm, and 130 nm technology nodes, with our modeling method and the conventional Pi-model technique. Then, the results obtained from these two methods have been compared with HSPICE simulation results. The obtained results show a better match in the propagation delay computations for global interconnects between our proposed model and HSPICE simulations, with respect to the conventional techniques such as Pi-model. According to the obtained results, the difference between our model and HSPICE simulations in the mentioned technology nodes is (0.29–22.92)%, whereas this difference is (11.13–38.29)% for another model.

Sign in / Sign up

Export Citation Format

Share Document