scholarly journals Dynamic Topology Model of Q-Learning LEACH Using Disposable Sensors in Autonomous Things Environment

2020 ◽  
Vol 10 (24) ◽  
pp. 9037
Author(s):  
Jae Hyuk Cho ◽  
Hayoun Lee
Keyword(s):  
Recent Decade ◽  
Q Learning ◽  
Adaptive Clustering ◽  
Dynamic Topology ◽  
Node Location ◽  
Dynamic Location ◽  
Topology Model ◽  
Strong Winds ◽  
Transmission Energy Consumption ◽  
The Internet Of Things

Low-Energy Adaptive Clustering Hierarchy (LEACH) is a typical routing protocol that effectively reduces transmission energy consumption by forming a hierarchical structure between nodes. LEACH on Wireless Sensor Network (WSN) has been widely studied in the recent decade as one key technique for the Internet of Things (IoT). The main aims of the autonomous things, and one of advanced of IoT, is that it creates a flexible environment that enables movement and communication between objects anytime, anywhere, by saving computing power and utilizing efficient wireless communication capability. However, the existing LEACH method is only based on the model with a static topology, but a case for a disposable sensor is included in an autonomous thing’s environment. With the increase of interest in disposable sensors which constantly change their locations during the operation, dynamic topology changes should be considered in LEACH. This study suggests the probing model for randomly moving nodes, implementing a change in the position of a node depending on the environment, such as strong winds. In addition, as a method to quickly adapt to the change in node location and construct a new topology, we propose Q-learning LEACH based on Q-table reinforcement learning and Fuzzy-LEACH based on Fuzzifier method. Then, we compared the results of the dynamic and static topology model with existing LEACH on the aspects of energy loss, number of alive nodes, and throughput. By comparison, all types of LEACH showed sensitivity results on the dynamic location of each node, while Q-LEACH shows best performance of all.

scholarly journals Deep Reinforcement Learning by Balancing Offline Monte Carlo and Online Temporal Difference Use Based on Environment Experiences

Symmetry ◽  
2020 ◽  
Vol 12 (10) ◽  
pp. 1685 ◽  
Author(s):  
Chayoung Kim
Keyword(s):  
Monte Carlo ◽  
Reinforcement Learning ◽  
Real Time ◽  
Temporal Difference ◽  
Q Learning ◽  
State Action ◽  
Proposed Model ◽  
Reward Functions ◽  
And Performance ◽  
The Internet Of Things

Owing to the complexity involved in training an agent in a real-time environment, e.g., using the Internet of Things (IoT), reinforcement learning (RL) using a deep neural network, i.e., deep reinforcement learning (DRL) has been widely adopted on an online basis without prior knowledge and complicated reward functions. DRL can handle a symmetrical balance between bias and variance—this indicates that the RL agents are competently trained in real-world applications. The approach of the proposed model considers the combinations of basic RL algorithms with online and offline use based on the empirical balances of bias–variance. Therefore, we exploited the balance between the offline Monte Carlo (MC) technique and online temporal difference (TD) with on-policy (state-action–reward-state-action, Sarsa) and an off-policy (Q-learning) in terms of a DRL. The proposed balance of MC (offline) and TD (online) use, which is simple and applicable without a well-designed reward, is suitable for real-time online learning. We demonstrated that, for a simple control task, the balance between online and offline use without an on- and off-policy shows satisfactory results. However, in complex tasks, the results clearly indicate the effectiveness of the combined method in improving the convergence speed and performance in a deep Q-network.

Structural Dynamic Topology Optimization of the Transmission Housing

2011 ◽  
Vol 308-310 ◽  
pp. 368-372
Author(s):  
Shou Wen Yao ◽  
Jian Li Lv ◽  
Qing Dong Peng
Keyword(s):  
Topology Optimization ◽  
Dynamic Performance ◽  
Cad Model ◽  
Mass Reduction ◽  
Element Analysis ◽  
Structural Dynamic ◽  
Design Variables ◽  
The Finite Element Analysis ◽  
Dynamic Topology ◽  
Topology Model

Dynamic performance is one of the most important factors in the product’s life. Transmission housing is one of the important components in vehicle, which has direct influence on the vehicle’s powertrain performance. Dynamic topology optimization can improve the product’s performance. The dynamic topology model is built, in which the density of elements are the design variables, the displacement of frequency response and volume are the constraints, and the objective is to maximize the first natural frequency of the housing. According to the result of optimization, the CAD model of housing is rebuilt and the finite element analysis of the new housing is done.The results show that both the static and dynamic performance are improved besides the mass reduction, namely, dynamic topology optimization can significantly improve the product’s performance.

scholarly journals Fog Fragment Cooperation on Bandwidth Management Based on Reinforcement Learning

Sensors ◽  
2020 ◽  
Vol 20 (23) ◽  
pp. 6942
Author(s):  
Motahareh Mobasheri ◽  
Yangwoo Kim ◽  
Woongsup Kim
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Fog Computing ◽  
Learning Approach ◽  
Data Generation ◽  
Bandwidth Management ◽  
Q Learning ◽  
Decision Making Problem ◽  
Iot Devices ◽  
The Internet Of Things

The term big data has emerged in network concepts since the Internet of Things (IoT) made data generation faster through various smart environments. In contrast, bandwidth improvement has been slower; therefore, it has become a bottleneck, creating the need to solve bandwidth constraints. Over time, due to smart environment extensions and the increasing number of IoT devices, the number of fog nodes has increased. In this study, we introduce fog fragment computing in contrast to conventional fog computing. We address bandwidth management using fog nodes and their cooperation to overcome the extra required bandwidth for IoT devices with emergencies and bandwidth limitations. We formulate the decision-making problem of the fog nodes using a reinforcement learning approach and develop a Q-learning algorithm to achieve efficient decisions by forcing the fog nodes to help each other under special conditions. To the best of our knowledge, there has been no research with this objective thus far. Therefore, we compare this study with another scenario that considers a single fog node to show that our new extended method performs considerably better.

scholarly journals Enhanced Energy-Efficient Heterogeneous Routing Protocols in WSNs for IoT Application

2019 ◽  
Vol 9 (1) ◽  
pp. 4418-4425
Keyword(s):  
Routing Protocols ◽  
Data Aggregation ◽  
Energy Efficient ◽  
Time Frame ◽  
Base Station ◽  
Sensor Nodes ◽  
Node Density ◽  
Adaptive Clustering ◽  
The Internet Of Things ◽  
Critical Structure

Bunch specific transducers of Wireless sensor networks (WSN) that give detecting administrations to the Internet of Things gadgets with constrained vitality and capacity assets. Because substitution or energizing of battery in tiny sensor nodes is practically incomprehensible, control utilization winds up one of the critical structure issues in WSN for the future, we proposed a crossbreed directing convention: Advanced Zone-Stable Election Protocol (AZ-SEP) with nature of heterogeneous WSNs for IoT situations. In this convention, a few nodes transmit information legitimately to the base station while some utilization the bunching method to send information to the base station. We actualized AZ-SEP and contrasted it and the customary Low Energy adaptive clustering hierarchy (LEACH). Recreation results demonstrated that Z-SEP improved the steadiness time frame and throughput than existing conventions like LEACH. The proposed AZ-SEP convention outflanks when contrasted with the current LEACH convention with a 64% ascent in better output in the form throughput and broadening the quantity of alive tiny nodes to 2702 rounds which can be utilized to improve the IoT lifetime. At the point when contrasted and other vitality productive conventions, it is discovered that the proposed calculation performs better as far as dependability period and system lifetime in various situations of region, vitality and node density. Thus our simulation result will show enhanced energy, throughput with data aggregation

scholarly journals Efficient Localized Data Aggregation of IoT Devices and its Application in Agriculture

2020 ◽  
Vol 8 (4S5) ◽  
pp. 259-264
Keyword(s):  
Optimal Path ◽  
Population Based ◽  
Social Environments ◽  
Routing Problem ◽  
Adaptive Clustering ◽  
Routing Optimization ◽  
Learning Procedure ◽  
The Social ◽  
Iot Devices ◽  
The Internet Of Things

The Internet of Things (IoT) has a likely future for all technologies that are associated to the life of humans. Any communication between the social environments along with the user contexts will be made through the smart interfaces. The IoT will have to link to different diverse devices found in the Wireless Sensor Network (WSN). Thus, the routing optimization which is energy efficient has become the primary factor in the performance of the network in the IoT. The widely popular routing used in WSN, Multi-hop Low Energy Adaptive Clustering Hierarchy (LEACH) protocol, is found to be energy inefficient. The work will deal with the choice of finding the optimal path in routing through meta heuristic techniques to improve the lifespan of the network and the energy efficiency of the network. There are different techniques of metaheuristics such as Teacher Learning Based Optimization (TLBO) and Particle Swarm Optimization (PSO) were used effectively for finding optimal solutions for problems. In this work, the PSO-based algorithms were used for locating an optimal sink position to their nodes that make the network efficient in terms of energy. The TLBO metaheuristic is population-based and is based on the concept of teaching and the learning procedure observed in a classroom, which is adapted to the routing problem. The results of the experiment prove that the proposed technique was achieved better levels of performance compared to the other methods..

scholarly journals Distributed Learning Based Joint Communication and Computation Strategy of IoT Devices in Smart Cities

Sensors ◽  
2020 ◽  
Vol 20 (4) ◽  
pp. 973
Author(s):  
Tianyi Liu ◽  
Ruyu Luo ◽  
Fangmin Xu ◽  
Chaoqiong Fan ◽  
Chenglin Zhao
Keyword(s):  
Smart Cities ◽  
Wireless Channel ◽  
Edge Computing ◽  
Resource Allocation Algorithm ◽  
Q Learning ◽  
Joint Policy ◽  
Low Efficiency ◽  
Learning Policies ◽  
Iot Devices ◽  
The Internet Of Things

With the development of global urbanization, the Internet of Things (IoT) and smart cities are becoming hot research topics. As an emerging model, edge computing can play an important role in smart cities because of its low latency and good performance. IoT devices can reduce time consumption with the help of a mobile edge computing (MEC) server. However, if too many IoT devices simultaneously choose to offload the computation tasks to the MEC server via the limited wireless channel, it may lead to the channel congestion, thus increasing time overhead. Facing a large number of IoT devices in smart cities, the centralized resource allocation algorithm needs a lot of signaling exchange, resulting in low efficiency. To solve the problem, this paper studies the joint policy of communication and computing of IoT devices in edge computing through game theory, and proposes distributed Q-learning algorithms with two learning policies. Simulation results show that the algorithm can converge quickly with a balanced solution.

scholarly journals The Internet of Things in Agriculture - the advantages and opportunities

2021 ◽  
Vol 13 (30) ◽  
pp. 53-63
Author(s):  
Margarita Gocheva ◽  
◽  
Velika Kuneva ◽  
Georgi Gochev ◽  
◽  
...  
Keyword(s):  
Internet Of Things ◽  
Smart Cities ◽  
Recent Decade ◽  
The Internet ◽  
Smart Meters ◽  
Significant Information ◽  
Complete Control ◽  
Measuring Devices ◽  
Wide Range ◽  
The Internet Of Things

The Internet of Things (IoT) has become increasingly popular in the recent decade. The Internet of Things helps people live and work smarter, as well as gain complete control over their lives. The concept of the IoT went widely into practice in different fields – Infrastructure, Production, Healthcare, Banks, Smart cities, Insurance, Media, Retail, Connected homes / Smart buildings, Agriculture, and many more. Modern agriculture can show its potential and importance by using these innovative technologies. The measuring devices, ensuring the transformation of data for the external environment into machine-readable data, at the same time filling the computing environment with significant information are very important. A wide range of measuring devices is used, from elementary sensors (e.g. temperature, pressure, illumination), consumption meters (e.g. smart meters) to complex integrated measuring systems. Satellites, drones, wireless sensor networks, analytical farming devices systems, farm management systems, big data applied to the farm are very useful and applicable in smart farming. The Internet of Things is a huge opportunity for farmers to monitor their crops and increase productivity, to monitor their livestock, to manage all the processes in their work and to take decisions at the right time. The article analyzes issues related to the modern IoT methods and their usage in general and in the area of agriculture. The main goal is to analyze the current state of IoT and its potential in areas of rural development and agriculture in the Republic of Bulgaria.

scholarly journals Learning nodes: machine learning-based energy and data management strategy

2021 ◽  
Vol 2021 (1) ◽  
Author(s):  
Yunmin Kim ◽  
Tae-Jin Lee
Keyword(s):  
Machine Learning ◽  
Learning Algorithm ◽  
Mac Protocol ◽  
Energy Resource ◽  
Machine Learning Algorithms ◽  
Medium Access ◽  
Q Learning ◽  
Use Of Resources ◽  
Viable Solution ◽  
The Internet Of Things

AbstractThe efficient use of resources in wireless communications has always been a major issue. In the Internet of Things (IoT), the energy resource becomes more critical. The transmission policy with the aid of a coordinator is not a viable solution in an IoT network, since a node should report its state to the coordinator for scheduling and it causes serious signaling overhead. Machine learning algorithms can provide the optimal distributed transmission mechanism with little overhead. A node can learn by itself by utilizing the machine learning algorithm and make the optimal transmission decision on its own. In this paper, we propose a novel learning Medium Access Control (MAC) protocol with learning nodes. Nodes learn the optimal transmission policy, i.e., minimizing the data and energy queue levels, using the Q-learning algorithm. The performance evaluation shows that the proposed scheme enhances the queue states and throughput.

Sign in / Sign up

Export Citation Format

Share Document