Edge computing servers like cloudlets from different service providers compensate scarce computational and storage resources of mobile devices, are distributed across access networks. However, the dynamically varying computational requirements of associated mobile devices make cloudlets either overloaded or under-loaded. Hence, load balancing among neighboring cloudlets appears to be an essential research problem. Especially, the load balancing problem among federated cloudlets from the same as well as different service providers for low-latency applications needs significant attention. Thus, in this paper, we propose a decentralized load balancing framework among federated cloudlets for low-latency applications that focuses on latency bound rather than latency minimization. In this framework, we employ dynamic processor slicing for handling heterogeneous classes of job requests. We propose a continuous-action reinforcement learning automata-based algorithm that enables cloudlets to independently compute the load balancing strategies in a completely distributed network setting without any exhaustive control message exchange. To capture the economic interaction among federated cloudlets, we model this load balancing problem as an economic and non-cooperative game and by scaffolding the properties of the game formulation, we achieve faster convergence of the reinforcement learning automata. Furthermore, through extensive simulations, we study the impacts of exploration and exploitation on learning accuracy.
On-line PID tuning for engine idle-speed control using continuous action reinforcement learning automata
