Achieving Effective Resource Management in Distributed SDN Controller Architectures

<p>As an emerging computer networking paradigm, Software-Defined Networking (SDN) empowers network operators with simplified network configuration and centralized network management. Recently, distributed controller architectures have become a notable invention where multiple controllers are jointly deployed in the network for request processing. One major research challenge for distributed controller architectures is to effectively manage the controller resources including allocating sufficient controllers to the suitable network locations and making the best use of the given controller resources. In general, existing approaches for managing the controller resources in the literature can be classified into three main directions. Designing new controller architectures belongs to the first direction, where the focus is on enabling workload shifting among controllers using switch migration. Designing controller placement algorithms to identify the number and locations of controllers is the second direction. Given the controller placement solution, the third direction is controller scheduling which aims to make the best use of the shared controllers by properly distributing requests among them. However, existing approaches have three major limitations. First, existing controller architectures feature a switch-controller binding which restricts the requests generated by a switch to only be processed by a predefined controller. Since each switch comes with different workload and the workload can be time-variant, the binding renders the bound controller susceptible to either being overloaded or underloaded. Second, existing placement algorithms have consistently underestimated the importance of controller scheduling. Due to the NP-hardness of the placement problem, Genetic Algorithm (GA) is a promising candidate. However, as a population-based approach, GA can be computationally expensive. Especially in a large network, the corresponding search space becomes too large for GA to handle effectively. Third, existing approaches for controller scheduling are mostly designed under the switch-controller binding constraint. When the scheduling is performed at a per-request level, the scheduling complexity increases significantly, rendering the efficiency and effectiveness of existing algorithms questionable. Apart from that, existing studies mainly focus on manually designing request dispatching policy which strongly relies on domain knowledge and involves a time-consuming fine-tuning process. The overall goal of this thesis is to effectively manage the controller resources in distributed SDN controller architectures. To address the three major limitations, three research objectives are established. First, this thesis aims to propose a new controller architecture to enable flexible controller placement and scheduling. Second, the thesis focuses on effectively and scalably identifying suitable controller placement while jointly taking the controller scheduling problem into consideration. Third, the thesis seeks to incorporate machine learning techniques in the request dispatching policy design to automatically learn adaptive and effective policies. To achieve the first objective, this thesis proposes a new BindingLess Architecture for distributed Controllers (BLAC) which features bindingless association between switches and controllers. With the newly introduced scheduling layer, requests can be transparently and flexibly dispatched among multiple controllers without invoking the time-consuming and complicated switch migration. Experiments conducted in this thesis show that BLAC significantly reduces the average response time and improves the throughput compared to existing SDN architectures. To achieve the second objective, this thesis proposes a Clustering-based Genetic Algorithm with Cooperative Clusters (CGA-CC) to tackle the controller placement problem. Particularly, CGA-CC partitions a large network into non-overlapping sub-networks to substantially reduce the search space of GA. Within each sub-network, GA is applied to identifying the placement solution. The quality of any given placement solution is evaluated by a gradient-descent-based scheduling algorithm which is developed to optimize the probability distribution of requests among all controllers. Moreover, a greedy load re-distribution mechanism is developed to handle unexpected demand variations by dynamically forwarding indigestible requests to adjacent sub-networks. Extensive simulations show that our algorithms can significantly outperform several existing and state-of-the-art algorithms and is more robust in handling unexpected traffic bursts. To achieve the third objective, this thesis proposes a Multi-Agent (MA) deep-reinforcement-learning-based approach with the aim to automatically learn adaptive, effective, and efficient policies used by each switch. In particular, a new adaptive policy representation is proposed to support networks with a changing number of controllers. To enable the training of an adaptive policy, a new policy gradient calculation technique is developed. Then the policy design problem is formulated as an MA Markov Decision Processing and a new MA training algorithm is proposed. The results show that the policy designed by our algorithm can easily adapt to networks with a changing number of controllers. Moreover, our policy can achieve significantly better performance compared with existing policies including the man-made policy (e.g., weighted round-robin), the model-based policy (e.g., the gradient-descent-based scheduling algorithm), and policies designed by other reinforcement learning algorithms (e.g., the proximal policy optimization algorithm).</p>

Achieving Effective Resource Management in Distributed SDN Controller Architectures

<p>As an emerging computer networking paradigm, Software-Defined Networking (SDN) empowers network operators with simplified network configuration and centralized network management. Recently, distributed controller architectures have become a notable invention where multiple controllers are jointly deployed in the network for request processing. One major research challenge for distributed controller architectures is to effectively manage the controller resources including allocating sufficient controllers to the suitable network locations and making the best use of the given controller resources. In general, existing approaches for managing the controller resources in the literature can be classified into three main directions. Designing new controller architectures belongs to the first direction, where the focus is on enabling workload shifting among controllers using switch migration. Designing controller placement algorithms to identify the number and locations of controllers is the second direction. Given the controller placement solution, the third direction is controller scheduling which aims to make the best use of the shared controllers by properly distributing requests among them. However, existing approaches have three major limitations. First, existing controller architectures feature a switch-controller binding which restricts the requests generated by a switch to only be processed by a predefined controller. Since each switch comes with different workload and the workload can be time-variant, the binding renders the bound controller susceptible to either being overloaded or underloaded. Second, existing placement algorithms have consistently underestimated the importance of controller scheduling. Due to the NP-hardness of the placement problem, Genetic Algorithm (GA) is a promising candidate. However, as a population-based approach, GA can be computationally expensive. Especially in a large network, the corresponding search space becomes too large for GA to handle effectively. Third, existing approaches for controller scheduling are mostly designed under the switch-controller binding constraint. When the scheduling is performed at a per-request level, the scheduling complexity increases significantly, rendering the efficiency and effectiveness of existing algorithms questionable. Apart from that, existing studies mainly focus on manually designing request dispatching policy which strongly relies on domain knowledge and involves a time-consuming fine-tuning process. The overall goal of this thesis is to effectively manage the controller resources in distributed SDN controller architectures. To address the three major limitations, three research objectives are established. First, this thesis aims to propose a new controller architecture to enable flexible controller placement and scheduling. Second, the thesis focuses on effectively and scalably identifying suitable controller placement while jointly taking the controller scheduling problem into consideration. Third, the thesis seeks to incorporate machine learning techniques in the request dispatching policy design to automatically learn adaptive and effective policies. To achieve the first objective, this thesis proposes a new BindingLess Architecture for distributed Controllers (BLAC) which features bindingless association between switches and controllers. With the newly introduced scheduling layer, requests can be transparently and flexibly dispatched among multiple controllers without invoking the time-consuming and complicated switch migration. Experiments conducted in this thesis show that BLAC significantly reduces the average response time and improves the throughput compared to existing SDN architectures. To achieve the second objective, this thesis proposes a Clustering-based Genetic Algorithm with Cooperative Clusters (CGA-CC) to tackle the controller placement problem. Particularly, CGA-CC partitions a large network into non-overlapping sub-networks to substantially reduce the search space of GA. Within each sub-network, GA is applied to identifying the placement solution. The quality of any given placement solution is evaluated by a gradient-descent-based scheduling algorithm which is developed to optimize the probability distribution of requests among all controllers. Moreover, a greedy load re-distribution mechanism is developed to handle unexpected demand variations by dynamically forwarding indigestible requests to adjacent sub-networks. Extensive simulations show that our algorithms can significantly outperform several existing and state-of-the-art algorithms and is more robust in handling unexpected traffic bursts. To achieve the third objective, this thesis proposes a Multi-Agent (MA) deep-reinforcement-learning-based approach with the aim to automatically learn adaptive, effective, and efficient policies used by each switch. In particular, a new adaptive policy representation is proposed to support networks with a changing number of controllers. To enable the training of an adaptive policy, a new policy gradient calculation technique is developed. Then the policy design problem is formulated as an MA Markov Decision Processing and a new MA training algorithm is proposed. The results show that the policy designed by our algorithm can easily adapt to networks with a changing number of controllers. Moreover, our policy can achieve significantly better performance compared with existing policies including the man-made policy (e.g., weighted round-robin), the model-based policy (e.g., the gradient-descent-based scheduling algorithm), and policies designed by other reinforcement learning algorithms (e.g., the proximal policy optimization algorithm).</p>

Achieving Balanced Load Distribution with Reinforcement Learning-Based Switch Migration in Distributed SDN Controllers

Distributed controllers in software-defined networking (SDN) become a promising approach because of their scalable and reliable deployments in current SDN environments. Since the network traffic varies with time and space, a static mapping between switches and controllers causes uneven load distribution among controllers. Dynamic migration of switches methods can provide a balanced load distribution between SDN controllers. Recently, existing reinforcement learning (RL) methods for dynamic switch migration such as MARVEL are modeling the load balancing of each controller as linear optimization. Even if it is widely used for network flow modeling, this type of linear optimization is not well fitted to the real-world workload of SDN controllers because correlations between resource types are unexpectedly and continuously changed. Consequently, using the linear model for resource utilization makes it difficult to distinguish which resource types are currently overloaded. In addition, this yields a high time cost. In this paper, we propose a reinforcement learning-based switch and controller selection scheme for switch migration, switch-aware reinforcement learning load balancing (SAR-LB). SAR-LB uses the utilization ratio of various resource types in both controllers and switches as the inputs of the neural network. It also considers switches as RL agents to reduce the action space of learning, while it considers all cases of migrations. Our experimental results show that SAR-LB achieved better (close to the even) load distribution among SDN controllers because of the accurate decision-making of switch migration. The proposed scheme achieves better normalized standard deviation among distributed SDN controllers than existing schemes by up to 34%.