scholarly journals SARL: A reinforcement learning based QoS-aware IoT service discovery model

2020 ◽  
Vol 71 (6) ◽  
pp. 368-378
Author(s):  
Selahattin Kosunalp ◽  
Kubilay Demir
Keyword(s):  
Reinforcement Learning ◽  
Service Discovery ◽  
Discovery Process ◽  
Energy Usage ◽  
Proposed Model ◽  
Enormous Amount ◽  
Simulation Results ◽  
Composition Process ◽  
Message Overhead ◽  
Usage Efficiency

AbstractThe IoT environment includes the enormous amount of atomic services with dynamic QoS compared with traditional web services. In such an environment, in the service composition process, discovering a requested service meeting the required QoS is a di cult task. In this work, to address this issue, we propose a peer-to-peer-based service discovery model, which looks for the information about services meeting the requested QoS and functionality on an overlay constructed with users of services versus service nodes, with probably constrained resources. However, employing a plain discovery algorithm on the overlay network such as flooding, or k-random walk could cause high message overhead or delay. This necessitates an intelligent and adaptive discovery algorithm, which adapts itself based on users’ previous queries and the results. To fill this gap, the proposed service discovery approach is equipped with a reinforcement learning-based algorithm, named SARL. The reinforcement learning-based algorithm enables SARL to significantly reduce delay and message overhead in the service discovery process by ranking neighboring nodes based on users’ service request preferences and the service query results. The proposed model is implemented on the OMNet simulation platform. The simulation results demonstrate that SARL remarkably outperforms the existing approaches in terms of message overhead, reliability, timeliness, and energy usage efficiency.

Genetic Based Reinforcement Learning Load Control for Smart Grids

2013 ◽  
Vol 860-863 ◽  
pp. 2423-2426
Author(s):  
Xin Li ◽  
Dan Yu ◽  
Chuan Zhi Zang
Keyword(s):  
Reinforcement Learning ◽  
Electric Charge ◽  
Smart Grids ◽  
Mathematic Model ◽  
Customer Participation ◽  
Load Control ◽  
Demand Side ◽  
Energy Usage ◽  
Take The Best ◽  
Simulation Results

As the improvement of smart grids, the customer participation has reinvigorated interest in demand-side features such as load control for domestic users. A genetic based reinforcement learning (RL) load controller is proposed. The genetic is used to adjust the parameters of the controller. The RL algorithm, which is independent of the mathematic model, shows the particular superiority in load control. By means of learning procedures, the proposed controller can learn to take the best actions to regulate the energy usage for equipments with the features of high comfortable for energy usage and low electric charge meanwhile. Simulation results show that the proposed load controller can promote the performance energy usage in smart grids.

Hierarchical Simulated Annealing-Reinforcement Learning Energy Management for Smart Grids

2013 ◽  
Vol 805-806 ◽  
pp. 1206-1209 ◽  
Author(s):  
Xin Li ◽  
Chuan Zhi Zang ◽  
Xiao Ning Qin ◽  
Yang Zhang ◽  
Dan Yu
Keyword(s):  
Simulated Annealing ◽  
Reinforcement Learning ◽  
Energy Management ◽  
Smart Grids ◽  
Mathematic Model ◽  
Trial And Error ◽  
Fuzzy Information ◽  
Energy Usage ◽  
Take The Best ◽  
Simulation Results

For energy management problems in smart grid, a hybrid intelligent hierarchical controller based on simulated annealing (SA) and reinforcement learning (RL) is proposed. The SA is used to adjust the parameters of the controller. The RL algorithm shows the particular superiority, which is independent of the mathematic model and just needs simple fuzzy information obtained through trial-and-error and interaction with the environment. By means of learning procedures, the proposed controller can learn to take the best actions to regulate the energy usage for equipments with the features of high comfortable for energy usage and low electric charge meanwhile. Simulation results show that the proposed load controller can promote the performance energy usage in smart grids.

scholarly journals Simulating bout-and-pause patterns with reinforcement learning

2019 ◽  
Author(s):  
Kota Yamada ◽  
Atsunori Kanemura
Keyword(s):  
Reinforcement Learning ◽  
Computational Models ◽  
Response Rate ◽  
Temporal Structure ◽  
Initiation Rate ◽  
Proposed Model ◽  
Simulation Results ◽  
Underlying Mechanisms ◽  
Experimental Findings ◽  
Three Components

AbstractAnimal responses occur according to a specific temporal structure composed of two states, where a bout is followed by a long pause until the next bout. Such about-and-pause pattern has three components: the bout length, the within-bout response rate, and the bout initiation rate. Previous studies have investigated how these three components are affected by experimental manipulations. However, it remains unknown what underlying mechanisms cause bout-and-pause patterns. In this article, we propose two mechanisms and examine computational models developed based on reinforcement learning. The model is characterized by two mechanisms. The first mechanism is choice—an agent makes a choice between operant and other behaviors. The second mechanism is cost—a cost is associated with the changeover of behaviors. These two mechanisms are extracted from past experimental findings. Simulation results suggested that both the choice and cost mechanisms are required to generate bout-and-pause patterns and if either of them is knocked out, the model does not generate bout-and-pause patterns. We further analyzed the proposed model and found that it reproduced the relationships between experimental manipulations and the three components that have been reported by previous studies. In addition, we showed alternative models can generate bout-and-pause patterns as long as they implement the two mechanisms.

scholarly journals Simulating bout-and-pause patterns with reinforcement learning

PLoS ONE ◽  
2020 ◽  
Vol 15 (11) ◽  
pp. e0242201
Author(s):  
Kota Yamada ◽  
Atsunori Kanemura
Keyword(s):  
Reinforcement Learning ◽  
Computational Models ◽  
Response Rate ◽  
Temporal Structure ◽  
Initiation Rate ◽  
Proposed Model ◽  
Simulation Results ◽  
Underlying Mechanisms ◽  
Experimental Findings ◽  
Three Components

Animal responses occur according to a specific temporal structure composed of two states, where a bout is followed by a long pause until the next bout. Such a bout-and-pause pattern has three components: the bout length, the within-bout response rate, and the bout initiation rate. Previous studies have investigated how these three components are affected by experimental manipulations. However, it remains unknown what underlying mechanisms cause bout-and-pause patterns. In this article, we propose two mechanisms and examine computational models developed based on reinforcement learning. The model is characterized by two mechanisms. The first mechanism is choice—an agent makes a choice between operant and other behaviors. The second mechanism is cost—a cost is associated with the changeover of behaviors. These two mechanisms are extracted from past experimental findings. Simulation results suggested that both the choice and cost mechanisms are required to generate bout-and-pause patterns and if either of them is knocked out, the model does not generate bout-and-pause patterns. We further analyzed the proposed model and found that it reproduced the relationships between experimental manipulations and the three components that have been reported by previous studies. In addition, we showed alternative models can generate bout-and-pause patterns as long as they implement the two mechanisms.

Intelligent Agent Based Model for Auction Service Discovery in Mobile E-Commerce

2012 ◽  
Vol 8 (1) ◽  
pp. 76-97
Author(s):  
Nandini S. Sidnal ◽  
Sunilkumar S. Manvi
Keyword(s):  
Web Service ◽  
Service Discovery ◽  
Intelligent Agent ◽  
Performance Parameters ◽  
Discovery Process ◽  
Agent Based ◽  
Web Service Discovery ◽  
Bdi Architecture ◽  
On Line ◽  
Simulation Results

Internet enabled auctions are one of the popular application which basically require a web service discovery mechanism that is efficient in all perspectives. This paper focuses on auction service discovery and building repository of services for the use of E-customers. The auction service directory (repository) is developed based on the customer’s desires. Agent based Belief Desire Intention (BDI) architecture is used in this model, not only to support the service discovery process in spotty or no connectivity network environment but also to automate the process so that it enables the mobile users to complete the discovery process successfully without continuous on-line presence. The simulation results depict that the performance parameters like customer satisfaction, availability of requested services and stability in fetching the services are better in the proposed service discovery model as compared to auction based advertisement facilitated service discovery mechanism.

Wear of a Tool in Double-Disk Lapping of Silicon Wafers

2010 ◽  
Author(s):  
Adam Barylski ◽  
Mariusz Deja
Keyword(s):  
Experimental Data ◽  
Integrated Circuits ◽  
Tool Wear ◽  
Silicon Wafers ◽  
Silicon Ingot ◽  
Proposed Model ◽  
Tool Wear Prediction ◽  
Simulation Results ◽  
High Degree ◽  
Kinematical Parameters

Silicon wafers are the most widely used substrates for fabricating integrated circuits. A sequence of processes is needed to turn a silicon ingot into silicon wafers. One of the processes is flattening by lapping or by grinding to achieve a high degree of flatness and parallelism of the wafer [1, 2, 3]. Lapping can effectively remove or reduce the waviness induced by preceding operations [2, 4]. The main aim of this paper is to compare the simulation results with lapping experimental data obtained from the Polish producer of silicon wafers, the company Cemat Silicon from Warsaw (www.cematsil.com). Proposed model is going to be implemented by this company for the tool wear prediction. Proposed model can be applied for lapping or grinding with single or double-disc lapping kinematics [5, 6, 7]. Geometrical and kinematical relations with the simulations are presented in the work. Generated results for given workpiece diameter and for different kinematical parameters are studied using models programmed in the Matlab environment.

Methods of Mathematical Modeling of the Leaching Process of Cobalt-Containing Solutions

2021 ◽  
Vol 316 ◽  
pp. 661-666
Author(s):  
Nataliya V. Mokrova
Keyword(s):  
Mathematical Modeling ◽  
Mathematical Model ◽  
Chemical Kinetics ◽  
Group Method ◽  
Data Handling ◽  
Multistage Processes ◽  
Leaching Process ◽  
Proposed Model ◽  
Simulation Results ◽  
The Mathematical Model

Current cobalt processing practices are described. This article discusses the advantages of the group argument accounting method for mathematical modeling of the leaching process of cobalt solutions. Identification of the mathematical model of the cascade of reactors of cobalt-producing is presented. Group method of data handling is allowing: to eliminate the need to calculate quantities of chemical kinetics; to get the opportunity to take into account the results of mixed experiments; to exclude the influence of random interference on the simulation results. The proposed model confirms the capabilities of the group method of data handling for describing multistage processes.

Collaborative Scheduling of Algorithms for Path Planning of Unmanned Systems

2021 ◽  
Vol 01 ◽  
Author(s):  
Ying Li ◽  
Chubing Guo ◽  
Jianshe Wu ◽  
Xin Zhang ◽  
Jian Gao ◽  
...  
Keyword(s):  
Genetic Algorithm ◽  
Reinforcement Learning ◽  
Path Planning ◽  
Unmanned Systems ◽  
Ant Colony Optimization Algorithm ◽  
A Algorithm ◽  
Collaborative Scheduling ◽  
Simulation Results ◽  
Planning Problems ◽  
Effective Path

Background: Unmanned systems have been widely used in multiple fields. Many algorithms have been proposed to solve path planning problems. Each algorithm has its advantages and defects and cannot adapt to all kinds of requirements. An appropriate path planning method is needed for various applications. Objective: To select an appropriate algorithm fastly in a given application. This could be helpful for improving the efficiency of path planning for Unmanned systems. Methods: This paper proposes to represent and quantify the features of algorithms based on the physical indicators of results. At the same time, an algorithmic collaborative scheme is developed to search the appropriate algorithm according to the requirement of the application. As an illustration of the scheme, four algorithms, including the A-star (A*) algorithm, reinforcement learning, genetic algorithm, and ant colony optimization algorithm, are implemented in the representation of their features. Results: In different simulations, the algorithmic collaborative scheme can select an appropriate algorithm in a given application based on the representation of algorithms. And the algorithm could plan a feasible and effective path. Conclusion: An algorithmic collaborative scheme is proposed, which is based on the representation of algorithms and requirement of the application. The simulation results prove the feasibility of the scheme and the representation of algorithms.

scholarly journals A strategy learning model for autonomous agents based on classification

2015 ◽  
Vol 25 (3) ◽  
pp. 471-482 ◽  
Author(s):  
Bartłomiej Śnieżyński
Keyword(s):  
Reinforcement Learning ◽  
Supervised Learning ◽  
Learning Process ◽  
Autonomous Agents ◽  
Good Alternative ◽  
Learning Model ◽  
Learning Method ◽  
Complex Environments ◽  
Agent Based ◽  
Proposed Model

AbstractIn this paper we propose a strategy learning model for autonomous agents based on classification. In the literature, the most commonly used learning method in agent-based systems is reinforcement learning. In our opinion, classification can be considered a good alternative. This type of supervised learning can be used to generate a classifier that allows the agent to choose an appropriate action for execution. Experimental results show that this model can be successfully applied for strategy generation even if rewards are delayed. We compare the efficiency of the proposed model and reinforcement learning using the farmer-pest domain and configurations of various complexity. In complex environments, supervised learning can improve the performance of agents much faster that reinforcement learning. If an appropriate knowledge representation is used, the learned knowledge may be analyzed by humans, which allows tracking the learning process

Sign in / Sign up

Export Citation Format

Share Document