On community structure validation in real networks

Community structure is a commonly observed feature of real networks. The term refers to the presence in a network of groups of nodes (communities) that feature high internal connectivity, but are poorly connected between each other. Whereas the issue of community detection has been addressed in several works, the problem of validating a partition of nodes as a good community structure for a real network has received considerably less attention and remains an open issue. We propose a set of indices for community structure validation of network partitions that are based on an hypothesis testing procedure that assesses the distribution of links between and within communities. Using both simulations and real data, we illustrate how the proposed indices can be employed to compare the adequacy of different partitions of nodes as community structures in a given network, to assess whether two networks share the same or similar community structures, and to evaluate the performance of different network clustering algorithms.

Crane Cloud: A Resilient Multi-Cloud Service Layer for Resource Constrained Settings

Whereas the main cloud providers have set up cloud services on stable infrastructure, developers and users situated in low-resource settings access cloud services and platforms using low-end computing devices that often connect to the Internet via slow mobile connections. These settings require custom software abstraction layers that consider such bandwidth constraints and intermittent connections as a rule rather than the exception. In this paper, we identify key challenges for developing for and accessing cloud services in resource constrained settings, namely, (1) Frequent Internet partitions and bandwidth constraints, (2) Data jurisdiction restrictions, (3) Vendor lock-in, and (4) Poor quality of service. To address these challenges, we propose a set of design considerations and properties for a resilient multi-cloud service layer, that includes: (1) Containerisation and orchestration of applications, (2) Service scheduling and replication, (3) Portability and multi-cloud migration, (4) Resilience to network partitions and bandwidth constraints, (5) Automated service discovery and load balancing, (6) Localised image registry, and (7) Support for platform monitoring and management. We present a prototype validation case study, Crane Cloud, an open source multi-cloud service abstraction layer built on-top of Kubernetes that is designed with inherent support for resilience to network partitions, microservice orchestration (deployment, scaling and management of containerized applications)a localized image registry, support for migration of services between private and public clouds to avoid vendor lock-in issues and platform monitoring. We evaluate the performance and user experience of Crane Cloud by implementing and deploying a computational and bandwidth intensive machine learning system shows lower response time compared when hosted on other public clouds.

ADMITS: Architecting Distributed Monitoring and Analytics in IoT-based Disaster Scenarios

The ADMITS project aims to develop algorithms, protocols and architectures to enable a distributed computing environment to provide support for monitoring, failure detection, and analytics in IoT disaster scenarios. We face a context where, every year, millions of people are affected by natural and man-made disasters, whereby governments all around the world spend huge amounts of resources on preparation, immediate response, and reconstruction. Recently, the Internet of Things (IoT) paradigm has been extensively used for efficiently managing disaster scenarios, such as volcanic disasters, floods, forest fire, land- slides, earthquakes, urban disasters, industrial and terrorists attacks, and so on. However, in a disaster scenario the communication/processing infrastructure and the devices themselves may fail, producing either temporary or permanent network partitions and loss of information. Moreover, it is expected that in the years to come, IoT will generate large amounts of data, making processing and analysis challenging in time-critical applications. Considering such challenges, ADMITS targets the development of a architecture in which IoT, Fog, and Cloud computing technologies participate to provide required capabilities for IoT data analytics, real-time stream processing, and failure monitoring for environments potentially subject to disasters. In this positional paper, we discuss the motivation, objectives, architecture, research challenges (and how to overcome them) and initial efforts for the ADMITS project.

Reduced network extremal ensemble learning (RenEEL) scheme for community detection in complex networks

We introduce an ensemble learning scheme for community detection in complex networks. The scheme uses a Machine Learning algorithmic paradigm we call Extremal Ensemble Learning. It uses iterative extremal updating of an ensemble of network partitions, which can be found by a conventional base algorithm, to find a node partition that maximizes modularity. At each iteration, core groups of nodes that are in the same community in every ensemble partition are identified and used to form a reduced network. Partitions of the reduced network are then found and used to update the ensemble. The smaller size of the reduced network makes the scheme efficient. We use the scheme to analyze the community structure in a set of commonly studied benchmark networks and find that it outperforms all other known methods for finding the partition with maximum modularity.

A Regularized Stochastic Block Model for the robust community detection in complex networks

The stochastic block model is able to generate random graphs with different types of network partitions, ranging from the traditional assortative structures to the disassortative structures. Since the stochastic block model does not specify which mixing pattern is desired, the inference algorithms discover the locally most likely nodes’ partition, regardless of its type. Here we introduce a new model constraining nodes’ internal degree ratios in the objective function to guide the inference algorithms to converge to the desired type of structure in the observed network data. We show experimentally that given the regularized model, the inference algorithms, such as Markov chain Monte Carlo, reliably and quickly find the assortative or disassortative structure as directed by the value of a single parameter. In contrast, when the sought-after assortative community structure is not strong in the observed network, the traditional inference algorithms using the degree-corrected stochastic block model tend to converge to undesired disassortative partitions.

Cluster Validity Measures Based Agglomerative Hierarchical Clustering for Network Data

The Louvain method is a method of agglomerative hierarchical clustering (AHC) that uses modularity as the merging criterion. Modularity is an evaluation measure for network partitions. Cluster validity measures are also used to evaluate cluster partitions and to determine the optimal number of clusters. Several cluster validity measures are constructed considering the geometric features of clusters. These measures and modularity are considered to be the same concept in the viewpoint of evaluating cluster partitions. In this paper, cluster validity measures based agglomerative hierarchical clustering (CVAHC) is proposed as a novel clustering method for network data. The cluster validity measures are used as a merging criterion and an evaluation measure for network data in the proposed method. Numerical experiments show that Dunn’s and Xie-Beni’s indices for network partitions are useful for network clustering.